End-to-End Vehicle Lateral Control with Transformer-Based Perception and Intention-Aware View Weighting

Abstract This study presents ViewSelective-CIL, an end-to-end intention aware multi view lateral control framework integrating multi-view semantic perception, Transformer-based attention, and high-level command–conditioned control. RGB images are fused with YOLOPv2-based semantic segmentation and encoded across left, front, and right views using a ViT-hybrid embedding. The key contribution is a ViewWeightGater module that dynamically blends HLC-driven attention with a Lane Density–based road complexity prior via a Hill function, producing view weights that are simultaneously intention-aware and structure-adaptive. Temporal dependencies and speed variation are incorporated through a GRU-based velocity encoder and a Temporal Transformer. Trained on 14,873 multi-view sequences from CARLA Town05, the model achieves over 95% autonomy in unseen urban environments (Town01 and Town02), exhibiting human-like attention allocation. Ablation studies confirm that semantic fusion and complexity-adaptive view weighting make distinct and complementary contributions to driving stability, while intervention-based metrics reveal behavioral differences that autonomy percentage alone does not fully capture. Project and dataset are available at https://github.com/donghyunkim39/ViewSelective-CIL