CrossFusion: a transformer-based architecture for multimodal 3D object detection

For autonomous systems that need accurate spatial awareness, three-dimensional object identification is essential. Modern systems mostly use LiDAR or camera data, which have limits in scene information acquisition. Recently, academics have studied multimodal methods. However, combining sensor inputs while detecting long-distance objects is still tricky. CrossFusion, a transformer-based multimodal 3D object identification system, addresses these challenges in this study. Two key innovations in our methodology are a hierarchical early fusion module that aligns and integrates LiDAR point cloud data with RGB image features and a transformer-based architecture with multi-scale deformable attention mechanisms for long-range dependency modeling. We have conducted a comprehensive analysis of the KITTI dataset to confirm the effectiveness of our approach. CrossFusion finds cars with average precision (AP) scores of 91.40% for easy, 85.14% for moderate, and 79.53% for hard. Our strategy improves long-range detection, a significant flaw in present methods. Ablation experiments show our multimodal fusion technique outperforms single-modality approaches. These findings demonstrate that CrossFusion lays the groundwork for 3D object detection, particularly in long-range perception applications, providing reassurance about its effectiveness.