A Method of Time Alignment in BEV Features for Multimodal Fusion Object Detection of Intelligent Vehicles

The fusion of images and point clouds for 3D object detection is a prominent research area in the field of intelligent vehicles. However, the temporal misalignment between image and point cloud modalities presents a critical challenge that affects the effectiveness of multimodal fusion, particularly in deep learning-based approaches. In this paper, a novel method is proposed to achieve time alignment of multimodal data. First, the disparities in data annotation caused by temporal misalignment are discussed, and their impact on object detection is also analyzed. Then, a time alignment model based on the hypothesis of linear translation in Bird’s Eye View (BEV) feature maps is introduced, providing a detailed design for the direction and distance of the translation operation. Finally, the validation experiments are conducted on the DAIR-V2X-V and nuScenes datasets, and the results prove that the model with the method proposed has significant performance improvements compared to the recent advanced models.