CRFusion: a novel LiDAR-camera fusion network for BEV map construction

The generation of high-definition semantic maps for the environment is a crucial component in automatic driving. Many methods based on different sensor fusion to achieve this key goal are proposed, and good performances have been obtained such as LiDAR and cameras. However, the issue of missing information in the part of maps still persists. To resolve the above problem, a novel LiDAR-Camera fusion network for Bird’s Eye View (BEV) map construction, CRFusion, is proposed. Firstly, the Long-Range LiDAR Feature Prediction (LRP) method is introduced, in which cross-attention is employed to enable interaction between image BEV features and LiDAR BEV features, and a self-attention mechanism is utilized to enhance the LiDAR BEV features. Secondly, to overcome the limitations of traditional fusion strategies on different lighting and weather conditions, the Feature Alignment and Dynamic Gated Fusion (DGF) methods are designed, which allow for spatial alignment of LiDAR and camera features. Fusion weights based on environmental conditions are adaptively adjusted, thereby the stability of the fusion effect is ensured. Finally, the feasibility of CRFusion is validated on the Mini NuScenes dataset and the experiment results conducted on the NuScenes public dataset demonstrate that the CRFusion method outperforms other methods. The superiority of the proposed method is thus confirmed.