A Novel Semantic Information Perception Architecture for Extreme Targets Detection in Complex Traffic Scenarios

Semantic perception information is essential for high-level tasks such as behavior prediction and path planning in autonomous vehicles. Traditional algorithms, when directly applied from computer vision to complex traffic scenarios, often struggle to adapt to certain challenging targets, including two typical kinds of extreme targets: the first includes those with partially missed features, commonly found in congested traffic scenes with occlusions or appearing suddenly in camera blind spots; the second encompasses targets with indistinct features, typically encountered in low-light conditions or at night. A mixed multi-scale attention module (MMSA) is designed to equilibrate parameter counts across various scales and bolster the network's capacity to learn incomplete features. The infrared imaging and a fusion algorithm are introduced to the detection algorithm's pre-processing part, enhancing the scene's thermal information in insufficient lighting. We conduct quantities of experiments on the well-lit KITTI, bimodal LLVIP and a self-collected datasets. The results show that mixed multi-scales attention module plays a significant role in detection of some specific objects and the fusion module help to improve the performance of networks especially in poor lighting scenarios.