PGEM-DETR: Physics-guided enhancement mechanism for drone-based object detection in adverse visual environments

Abstract Drone-based object detection in adverse visual environments (AVEs) poses significant challenges that remain largely unaddressed by existing methods. Degraded image quality from fog and low-light conditions substantially impairs the detection performance of conventional algorithms optimized for ideal conditions. In this paper, we propose Physics-Guided Enhancement Mechanisms with Detection Transformer (PGEM-DETR), a novel framework tailored for drone-based object detection in AVEs. Our approach integrates physics-guided enhancement mechanisms with the DETR architecture, combining environmental prior knowledge with deep learning techniques to overcome visual degradation challenges. Extensive experiments on our comprehensive AVE-CARPK, AVE-DroneVehicle, and AVE-VisDrone datasets demonstrate that PGEM-DETR significantly outperforms state-of-the-art object detectors under adverse conditions while maintaining comparable computational efficiency. PGEM-DETR-M achieves 83.1% AP$_5{}_0$ and 58.1% AP$_5{}{}_0$:$_9{}_5$ on the AVE-CARPK dataset, surpassing YOLOV12-X by 2.7% and 2.3%, respectively, while requiring only 73% of its parameters and 69% of its computational complexity. Our lightweight PGEM-DETR-N achieves 142.7 FPS with INT8 quantization on Nvidia Jetson Orin NX, showing strong potential for UAV deployment, though real-world validation beyond simulated adverse environments is still required.