Efficient transformer-based UAV image object detection network based on convolutional self-attention and matchability-adaptive classification

Abstract Accurate detection of small objects in complex low-altitude scenes is crucial for the widespread deployment of unmanned aerial vehicles (UAVs). However, UAV imagery often suffers from small object scales, cluttered backgrounds, and challenging feature extraction. To address these issues, we propose CSMA-DETR, an efficient transformer-based UAV object detection network with convolutional self-attention and matchability-adaptive classification. First, we design a novel feature extractor, Cross-Spatial Emulating Self-Attention with Convolution (CESC), which jointly integrates convolutional self-attention with spatial-domain processing. By using shared large kernels and hierarchical dynamic kernels, CESC emulates long-range dependency modeling and instance-adaptive behaviors of self-attention, strengthening multi-scale representations while reducing computational cost. Second, we introduce the Linear Patch-Aware Attention Feature Interaction (LPAFI) module. A multi-branch patch-aware attention mechanism captures cross-scale local correspondences, and a multi-scale linear attention mechanism further fuses CNN-style local detail representation with global context modeling, enabling efficient integration of local and global features. Finally, we propose Matchability-Varifocal Loss (MVL) to better optimize low-quality matched samples. MVL combines IoU-aware and matchability-aware weighting to dynamically adjust loss contributions and strengthen supervision for low-quality samples, accelerating convergence and improving robustness. Experiments on VisDrone2019 show that CSMA-DETR significantly outperforms the baseline, improving mAP50 and mAP50:95 by 5% and 2%, respectively, while reducing parameters and computation by 7.0% and 4.6%. Additional evaluations on DIOR and HIT-UAV further verify its effectiveness in cluttered scenes with densely distributed small objects, providing an efficient solution for real-time UAV detection.