Pedestrian Detection Algorithm Based on Multi-Scale Fusion and Dual-Backbone

Addressing the limitations of visible light singlemodal target detection algorithms, which are affected by external factors like illumination, and infrared single-modal algorithms, which lack complete target information, we propose a pedestrian detection algorithm based on dual-backbone and multi-modal multi-scale fusion, building upon YOLOv8.The network employs a dual-backbone structure derived from the YOLOv8n lightweight network. An SMMF(Scale-Multiple Multimodal Feature Fusion) module is designed to fuse features from both modalities at multiple scales. The GhostConv module is introduced to enrich features and reduce computational cost. Additionally, the SBAp2 is integrated into the network neck, enhancing the fusion of deep and shallow features and leveraging both deep semantic information and shallow image location information. The ShapeIoU is adopted as the loss function, replacing the original bounding box regression loss, to incorporate the shape and scale of the bounding box into the loss calculation, thereby improving target detection accuracy. Experimental results indicate that the improved algorithm achieves an AP0.5 of 78.5% and an FPS of 156.3. Compared to the original visible light single-modal and infrared single-modal YOLOv8n algorithms, the missed detection rates were reduced by 30.1% and 13.4%, respectively, while the AP0.5 is increased by 21.2% and 8.2%, demonstrating its superiority over the single-modal YOLOv8n detection algorithm.