3D vehicle detection improving method fusing image and LiDAR in foggy environment

3D vehicle detection still remains challenging due to the occlusion, illumination and special weather. This paper proposes a data-level fusion method to combine the images and LiDAR to achieve 3D vehicle detection in the foggy environment. This method first restores the corrupted multi-modal data though image defogging and point cloud defogging. Then the pseudo-LiDAR is generated by back-projecting the image pixels with the estimated image depth. After revisitation and transformation, the pseudo-LiDAR and LiDAR data can be fused in the 3D space. Finally, the fused point clouds are fed into the BtcDet detection network to obtain detection results. Furthermore, in order to evaluate 3D detection methods, this paper establishes a multi-modal dataset (images and LiDAR) of the foggy scenarios by simulation algorithms to compensate for the lack of foggy dataset. Based on the synthetic dataset, the experiments are conducted to evaluate the proposed methods and other comparative methods. The accuracy of the 3D target detection algorithm is higher than comparative methods.