Dual-modality BEV fusion with feature reconstruction for 3D object detection

Abstract Camera images and LiDAR point clouds provide complementary information for 3D object detection by supplying visual cues and geometric measurements, respectively. Existing bird’s-eye-view (BEV) fusion methods typically map modality-specific features into a shared BEV space and integrate them into a joint representation. Ideally, the fused representation should retain information from both modalities while distinguishing object regions from the surrounding background. However, in many existing BEV fusion methods, the fused representation is optimized primarily through the downstream detection objective, without direct constraints on modality information retention or foreground–background discrimination at the fusion stage. To remedy this shortcoming, we present a Camera–LiDAR 3D object detection framework with two complementary forms of auxiliary supervision for the fused BEV representation. Specifically, Dual-Modality BEV Feature Reconstruction uses two separate decoders to reconstruct the pre-fusion camera and LiDAR BEV features from the fused representation. The corresponding losses encourage modality-specific information to remain recoverable after fusion. In parallel, BEV Foreground Supervision derives class-agnostic targets from the BEV footprints of ground-truth 3D boxes and uses them to supervise a dense foreground map predicted from the fused feature, encouraging clearer foreground–background discrimination. Experiments on nuScenes show that the proposed method achieves 73.4% NDS and 70.8% mAP on the validation split, together with 74.6% NDS and 71.9% mAP on the test split. Ablation studies further confirm that each objective improves the baseline individually, while their combination delivers the best performance among the evaluated configurations.