End-to-end Object Detection System Using Multi-Source Data Fusion for Autonomous Driving

With the rapid development of autonomous driving, modern vehicles are equipped with multiple sensors. Data from these various sensors assist and cross-reference each other, enhancing autonomous driving capabilities. However, the efficient and cost-effective utilization of the vast amounts of data generated by these sensors remains a significant challenge. This study presents an end-to-end multi-source data object detection system based on the YOLOv8 model. By introducing the Pixel Cluster Rotation (PCR) and hint layer, the system enables the fusion of multi-source data without significantly increasing resource requirements. Evaluations show that compared to vision-only object detection, these methods allow the fusion of LiDAR and RGB data with approximately 0.38% additional neural network parameters, resulting in a 2.9% increase in the system's mean Average Precision (mAP). This research can be applied to low-cost platforms with limited system resources, promoting the broader adoption of autonomous driving technology.