Research on Object Detection in Complex Scenarios Based on <scp>ASA</scp>‐<scp>YOLOv5</scp>

The applications of target detection in complex scenarios cover a wide range of fields, such as pedestrian and vehicle detection in self‐driving cars, face recognition and abnormal behavior detection in security monitoring systems, hazardous materials safety detection in public transportation, and so on. These applications demonstrate the importance and the prospect of wide application of target detection techniques in solving practical problems in complex scenarios. However, in these real scenes, there are often problems such as mutual occlusion and scale change. Therefore, how to accurately identify the target in the real complex scenarios has become a big problem to be solved. In order to solve the above problem, the paper proposes a novel algorithm, Adaptive Self‐Attention‐YOLOv5 (ASA‐YOLOv5), which is built upon the YOLOv5s algorithm and demonstrates effectiveness for target identification in complex scenarios. First, the paper implements a fusion mechanism between the trunk and neck networks, enabling the fusion of features across different levels through upsampling and downsampling. This fusion process mitigates detection errors caused by feature loss. Second, the Shuffle Attention mechanism is introduced before upsampling and downsampling to suppress noise and amplify essential semantic information, further enhancing target identification accuracy. Lastly, the Adaptively Spatial Feature Fusion (ASFF) module and Receptive Field Blocks (RFBs) module are added in the head network, and it can improve feature scale invariance and expand the receptive field. The ability of the model to detect the target in the complex scene is improved effectively. Experimental results indicate a notable improvement in the model's mean Average Precision (mAP) by 2.1% on the COCO dataset and 0.7% on the SIXray dataset. The proposed ASA‐YOLOv5 algorithm can enhance the effectiveness for target detection in complex scenarios, and it can be widely used in real‐world settings. © 2024 Institute of Electrical Engineers of Japan and Wiley Periodicals LLC.