Enhancing YOLO for occluded vehicle detection with grouped orthogonal attention and dense object repulsion

In real-life complex traffic environments, vehicles are often occluded by extraneous background objects and other vehicles, leading to severe degradation of object detector performance. To address this issue, we propose a method named YOLO-OVD (YOLO for occluded vehicle detection) and a dataset for effectively handling vehicle occlusion in various scenarios. To highlight the model attention in unobstructed region of vehicles, we design a novel grouped orthogonal attention (GOA) module to achieve maximum information extraction between channels. We utilize grouping and channel shuffling to address the initialization and computational issues of original orthogonal filters, followed by spatial attention for enhancing spatial features in vehicle-visible regions. We introduce a CIoU-based repulsion term into the loss function to augment the network's localization accuracy in scenarios involving densely packed vehicles. Moreover, we explore the effect of the knowledge-based Laplacian Pyramid on the OVD performance, which contributes to fast convergence in training and ensures more detailed and comprehensive feature retention. We conduct extensive experiments on the established occluded vehicle detection dataset, which demonstrates that the proposed YOLO-OVD model significantly outperforms 14 representative object detectors. Notably, it achieves improvements of 4.7% in Precision, 3.6% in [email protected], and 1.9% in [email protected]:0.95 compared to the YOLOv5 baseline.