RCDFNet: A 4-D Radar and Camera Dual-Level Fusion Network for 3-D Object Detection

4-D radar, as an advanced vehicle sensor, provides denser point clouds and elevation information than millimeter-wave radar, establishing it as a valuable sensor for autonomous driving. Recently, the fusion of 4-D radar and camera has become a viable alternative to LiDAR for autonomous driving perception systems. However, existing fusion methods have not fully realized the potential of complementary advantages offered by these two modalities. To address this limitation, our study proposes RCDFNet, a dual-level fusion network for 3D object detection with 4-D radar and camera. Specifically, we first exploit the geometric information from the 4-D radar to extract semantic image features from the perspective view, yielding pseudo-camera features under the bird’s-eye view. In the first fusion level, the pseudo-camera features are integrated with the 4-D radar features using the spatial point fusion module to obtain the radar-guide fusion features. In the second fusion level, we employ the deformable attention mechanism to facilitate the interaction between camera-view-transform features and the radar-guide fusion features, to generate BEV-match fusion features. Furthermore, we propose the target-aware refinement module to learn of foreground targets’ occupancy adaptively in the bird’s-eye view, mitigating the impact of the background noise of two sensors. We validate the RCDFNet on the TJ4DRadSet and View-of-Delft (VoD) datasets. Experimental results demonstrate that RCDFNet significantly outperforms the baseline RCFusion, achieving state-of-the-art performance in 3D object detection. Code and models are available at https://github.com/D-Hourse/RCDFNet/tree/master.