Enhance Image BEV Features with LiDAR Gradient Guidance for 3D Object Detection

Multi-modal fusion 3D object detection is essential for autonomous driving perception. Recently, Bird's Eye View (BEV)-based methods have become the mainstream paradigm. Albeit remarkable progress, the performance of the multi-modal fusion model is limited by the unsatisfactory image BEV features, due to lacking depth information in 2D images. To address this issue, this paper first simply introduces a more advanced depth network to optimize the image BEV features. More importantly, this work proposes an Image Enhancement Module (IEM) to improve the image BEV features, guided by LiDAR gradients. Specifically, IEM first generates the Enhancement Mask (EM) by leveraging LiDAR gradients, then EM is multiplied with the Foreground Mask (FM) to produce the Foreground Enhancement Mask (FEM), which reweights the gradients of the image BEV features. Extensive experiments demonstrate the effectiveness of our methods. With the improved depth network and IEM, the multi-modal fusion model can attain superior detection results, with 69.0% mAP and 71.7% NDS on the nuScenes validation set, and 70.2% mAP and 72.5 % NDS on the nuScenes test set, respectively. Besides, IEM has no learnable parameters and does not affect inference efficiency.