Multi-Modal Fusion for 3D Object Detection in Autonomous Driving: A Review

Autonomous driving technologies have seen significant advancements due to improvements in artificial intelligence and sensor systems. A crucial element for safe and effective autonomous driving is the environmental perception system, with 3D object detection serving as a core task for vehicle awareness and decision-making. However, single-modality sensors, such as cameras, LiDAR, and radar, each have inherent limitations. Cameras provide rich semantic data but struggle under adverse conditions, LiDAR offers precise geometric data but is costly, and radar, while robust in all weather, lacks spatial resolution. Multi-modal sensor fusion addresses these challenges by combining complementary data from different sensor types, leading to a more comprehensive environmental understanding. This paper surveys the latest advancements in 3D object detection using multi-modal fusion for autonomous driving, focusing on feature-level fusion techniques, particularly the employment of Bird's-Eye-View (BEV) representation and Transformer architectures. The paper also explores the development of datasets, the evolution of fusion strategies, and the critical challenges faced, such as data alignment, performance efficiency, and robustness in harsh conditions. Finally, it outlines the future research directions aimed at optimizing the fusion process, including lightweight architectures and the integration of foundation models for improved performance and generalization.