Edge artificial intelligence and super-resolution for enhanced weapon detection in video surveillance

The prevalence of crimes involving handguns and knives underscores the importance of early weapon detection. This, along with the spread of video surveillance systems, boosted the development of automatic approaches for weapon detection from surveillance cameras. Despite the advancements from classical computer vision to Deep Learning (DL) techniques, accurately detecting weapons in real-time remains challenging due to their small size. Current DL methods, which attempt to mitigate this issue using complex detection architectures, are resource-intensive, resulting in high costs and energy usage, and hindering their deployment on efficient edge devices. This creates challenges in resource-limited environments, making these methods impractical for edge and real-time applications. To address these shortcomings, our work proposes YOLOSR, which integrates a You Only Look Once (YOLO) v8-small model with an Enhanced Deep Super Resolution (EDSR)-based network using a shared backbone. During training, the auxiliary Super Resolution (SR) helps in learning better features, which could benefit the weapon detection task. During inference, the SR branch is removed, keeping the detector’s computational complexity unchanged. The YOLOSR’s accuracy and efficiency were validated on our WeaponSense dataset and on a NVIDIA Jetson Nano, against other weapon detectors. The results exhibited that YOLOSR, compared to the state-of-the-art YOLOv8-small model, maintained the same computational complexity with 28.8 billion floating point operations and on-device latency of 101 ms per image, while increasing the Average Precision by 10.2 percentage points. Thus, the YOLOSR emerges as an effective solution for real-time weapon detection in resource-constrained environments, achieving an optimal trade-off between efficiency and accuracy. • YOLOSR merges Super-Resolution (SR) with YOLOv8 to enhance small weapon detection. • SR boosts training and is discarded at inference, with no extra computational cost. • YOLOSR achieved a 10.2% AP gain compared to YOLOv8, keeping the same real-time speed. • The approach runs at 101ms/image on Jetson Nano, enabling real-time edge inference. • The approach was validated on WeaponSense, a realistic video-surveillance dataset.