Multimodal sensor fusion with cross-modal alignment and attention mechanism for enhanced object detection in autonomous driving systems

LiDAR, radar, and cameras are widely used in autonomous driving systems, but each modality has inherent limitations, such as LiDAR's sensitivity to adverse weather, radar's low spatial resolution, and cameras' dependence on lighting conditions. To address these challenges, this study proposes a novel multi-modal fusion framework that integrates these sensors to enhance object detection accuracy and robustness. A cross-modal feature alignment strategy ensures spatial and semantic consistency across sensor data, while an attention-based mechanism dynamically adjusts the contributions of each modality based on their reliability in different scenarios. Experimental results on the KITTI and nuScenes datasets show that the framework achieves a mean Average Precision (mAP) of 89.4% while maintaining real-time efficiency at 36.8 FPS. Compared to single-modality baselines and traditional fusion methods, the proposed framework demonstrates superior detection performance, particularly in scenarios involving occlusion, low-light conditions, and dense traffic. Ablation studies validate the effectiveness of the cross-modal alignment and attention mechanisms, highlighting their critical roles in achieving robust detection. The proposed framework offers a scalable and efficient solution for autonomous driving systems, effectively addressing the limitations of single-modality sensors.