A Heterogeneous Data-Driven Multi-Sensor Collaborative Small Target Detection Method for Road Safety in Bad Weather

Autonomous driving (AD) systems requires multisensor collaboration to address challenges caused by complex weather scenarios. The recognition accuracy of autonomous driving system based on single resource, either on image only or Lidar only, becomes unreliable due to untested weather conditions, occlusions objects, and other factors. This paper proposes a decision-level fusion network based on an improved YOLOV7 and an improved CenterPoint network to build a multi-sensor-collaboration scheme. The overall accuracy of the proposed fusion algorithm is improved for small targets by adding multi-scale and multi-stage attention channel modules into the backbones of image recognition network and point cloud recognition network respectively. Moreover, the fusion algorithm introduces mixed distance constraints as the loss function for overlapping targets. The proposed fusion algorithm has been successfully tested on the public ONCE dataset mixed with a self-built dataset under various road conditions such as sunny, night, and rainy weather. The mAP of proposed decision-level fusion algorithm achieves <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\mathbf{8 3. 5 \%}$</tex> in sunny daytime, <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\mathbf{8 0 \%}$</tex> during nighttime and 79.1 % during rain time.