YOLOv8 Based Vehicle Object Detection Using Two-Stage Multi-Modality Fusion

In response to the issue of degraded detection performance of vehicle targets in autonomous driving scenarios due to varying degrees of occlusion and scale changes, this paper proposes a robust decision-level multimodal fusion object detection method. Firstly, natural images captured by camera, sparse depth maps (DM) and bird's eye view maps (BEVM) generated from the LiDAR point cloud information are separately fed into the end-to-end object detector YOLOv8. The employed YOLOv8 incorporates improved regression losses via Wise-IOU loss, for target position regression and classification within their respective modalities. Secondly, to address the problem of poor detection performance for heavily occluded targets, a K-nearest neighbor multi-output regressor (MOR) algorithm is used to transform BEV detection boxes into the frontal view. A two-stage fusion approach based on soft non-maximum suppression (soft-NMS) is adopted to fuse detection results from the three modalities (natural images, DM and BEVM). The first fusion is conducted by combining results from color images and DM, and the result is further fused with the transformed BEV detection results. The experimental results on the KITTI dataset demonstrate that the proposed method exhibits comprehensive advantages in both detection accuracy and speed compared to various other detection methods and specifically improves the detection performance for heavily occluded targets.