Spatial Reasoning and Risk Assessment for Autonomous Vehicles on Consumer Electronics Platforms Using a Customized Vision–Language Model With Data Augmentation
Document Type
Article
Publication Date
2-1-2026
Abstract
Accurate spatial reasoning and risk assessment from monocular video on consumer electronics platforms are prerequisites for safe decision-making in autonomous vehicles, yet general-purpose vision–language model (VLM) remains unreliable at lane-level localization and temporally grounded risk estimation. We present a deployment-oriented, spatially enhanced VLM that learns implicit 3D reconstructive information from monocular sequences via a reconstructive 3D tokenization pipeline and a spatial–visual fusion encoder built on MobileVLM. Concretely, the model fuses 3D reconstructive tokens with CLIP visual tokens through cross-attention to produce 3D-aware visual tokens, enabling lane-level localization and depth-aware reasoning about object orientation and inter-object relations. The unified decoder jointly generates scenario descriptions, object identities and positions, and converts spatial estimates into interpretable multi-level risk scores using a Time-to-Collision (TTC) mapping. Experimental results show that our system outperforms strong VLM baselines for multi-level risk classification on the nuScenes dataset. The generalization in CARLA and CoVLA further corroborates robust spatial reasoning and risk assessment. The findings indicate that coupling a lightweight VLM with monocular 3D cues via cross-attentional spatial–visual fusion yields accurate spatial localization and interpretable, transferable risk estimation that is robust under shift and practical for edge deployment in downstream tasks.
Keywords
autonomous vehicles, consumer electronics platform, risk assessment, safety-critical scenario, Vision–language model
Publication Title
IEEE Transactions on Consumer Electronics
ISSN
0098-3063
DOI
10.1109/TCE.2026.3655159
Recommended Citation
Song, Ziyu; Yang, Jing; Fang, Lei; Ali, Muhammad Umair; Kumar, Gyanendra; Bashir, Ali Kashif; Alturki, Nazik; Por, Lip Yee; and Lee, Seung Won, "Spatial Reasoning and Risk Assessment for Autonomous Vehicles on Consumer Electronics Platforms Using a Customized Vision–Language Model With Data Augmentation" (2026). Research Publications (2026 to 2030). 1403.
https://knova.um.edu.my/research_publications_2026_2030/1403
Funders
Ministry of Education - Kingdom of Saudi Arabia, Ministry of Science & ICT, Republic of Korea (RS [2024-0040 (5650), 059735, 2019-II19 (0421), RS [2025-2544 (3209), RS [2024-0044 (0881), NRF [2021-R1-I1A2)
Volume
72
Issue
1
First Page
2321
Last Page
2330
Publisher
Ministry of Education - Kingdom of Saudi Arabia, Ministry of Science & ICT, Republic of Korea (RS [2024-0040 (5650), 059735, 2019-II19 (0421), RS [2025-2544 (3209), RS [2024-0044 (0881), NRF [2021-R1-I1A2), Princess Nourah Bint Abdulrahman University (PNURSP2026R333)