BSM-NET: multi-bandwidth, multi-scale and multi-modal fusion network for 3D object detection of 4D radar and LiDAR

Abstract In recent years, with the rapid advancement of autonomous driving technology, the requirements for environmental perception tasks have become increasingly important. Four-dimensional (4D) millimeter-wave radar, an economical and reliable technology, has begun to attract attention. Additionally, LiDAR can measure accurately and is not easily interfered with, so it is also very popular. To overcome the limitations of single-sensor perception, this paper proposes the BSM-NET method, a multi-bandwidth, multi-scale, multi-modal fusion approach for 4D radar and LiDAR. This paper uses image technology to clean point cloud data, reducing errors and noise. BSM-NET consists of two key modules: multi-bandwidth fusion (MBF) and multi-scale fusion (MSF). MBF enhances data quality by capturing point cloud density and addressing issues such as gap filling and noise reduction. MSF improves accuracy and robustness through high-precision calculations. For better integration, we use the radar and LiDAR based multi-modal fusion (RLMF), which enables different types of data to learn from each other, allowing us to effectively fuse 80-line LiDAR and 4D radar data. Experimental results demonstrate that BSM-NET significantly outperforms the current state-of-the-art algorithms on Dual-Radar dataset. Compared to M2-Fusion, our fusion method achieves notable performance improvements of 2.04% and 2.21% in medium-difficulty 3D object detection and bird’s eye view detection for the vehicle category, respectively.