Bidirectional Radar–Camera Fusion With Dual Cross-Attention for 3-D Object Detection

3D object detection is one of the core technologies for autonomous driving. Relying on a single type of sensor is inadequate for effectively perceiving the driving environment, and multi-sensor fusion solutions mitigate this limitation. This article presents a novel approach for early and middle multimodal fusion, referred to as Bi-RC Fusion, that not only maximizes the utilization of raw data from heterogeneous sensors in the early fusion, but also facilitates bi-directional assistance between radar and camera through our module in the middle fusion stage, which consists of two parallel sets of multi-head cross-attention. In the final fusion stage, we have developed an adaptive feature fusion module to enhance the efficiency of cross-modal integration. Additionally, we have designed an efficient multi-scale feature fusion module to learn the fused feature representations across different scales. We have conducted extensive experiments on the nuScenes dataset, including robustness experiments particularly designed for challenging weather conditions. The experimental results demonstrate that our proposed method significantly enhances the performance of 3D object detection, particularly regarding metrics related to velocity and attribute errors.