Deep Learning Model for Multimodal Data Fusion in Autonomous Driving

This paper proposes a multimodal fusion network (DAMFNet) based on dynamic attention mechanism, which aims to solve the problem of efficient fusion of multi-source sensor data in autonomous driving. DAMFNet eliminates the spatiotemporal misalignment between different modalities through the spatiotemporal feature alignment module, and designs a dynamic attention module to adaptively adjust the weights of each modality according to the complexity of the scene. Experiments are carried out on nuScenes and KITTI datasets. The results show that DAMFNet achieves 89.2% mAP in the target detection task, which is 4.1% higher than the existing method BEV Former; in extreme weather scenarios, the mean translation error (mATE) is reduced by 23.5%. Ablation experiments verify the key role of the dynamic attention module, which improves the prediction accuracy of the model by 6.7% in scenes with changing lighting. Real-time tests show that DAMFNet achieves an inference speed of 120 FPS on the NVIDIA Orin platform, meeting the real-time requirements of autonomous driving. The research results provide a new solution for the application of multimodal fusion algorithms in complex traffic environments.