LT-DETR: lightweight UAV object detection and dual knowledge distillation for remote sensing scenarios

Abstract In recent years, the rapid advancement of unmanned aerial vehicle (UAV) technology has amplified the need for robust object detection algorithms capable of handling aerial perspectives in remote sensing applications. Traditional detection algorithms are often difficult to simultaneously cope with two challenges in UAV application scenarios, i.e. small target detection and lightweighting. To address these issues, we propose lightweight transformer-based detector, a lightweight object detector designed specifically for UAV-based object detection in remote sensing scenarios. To solve the problem of difficult feature extraction for small targets, we propose three modules to optimize feature fusion, namely Cross Stage Partial-Omni Kernel, dynamic sampling (DySample), and Lightweight Group Convolution Channel Shuffle. In order to improve the real-time operation of the model, we adopt a dual knowledge distillation strategy that combines feature distillation and logical distillation, which maintains the original accuracy while the model is lightweight. Experimental results on the VisDrone 2019 and TinyPerson datasets show that Our model achieves state-of-the-art performance while decreasing the network parameters and computation by nearly 50%.