RTS-DETR: Efficient Real-Time DETR for Small Object Detection

In recent years, object detection models DEtection TRansformer (DETR) series based on Transformer architecture have played a huge role in various fields. However, the DETR series models are not satisfactory in small object detection. Mainly due to the huge amount of calculation of DETR, a lot of feature information will be lost in the feature fusion stage and the low tolerance of small objects to Intersection over Union (IoU). In order to solve the above problems, we propose a near real-time detection model RTS-DETR. In this paper, we revisit Real-Time DEtection TRansformer (RT-DETR), which effectively handles multi-scale features by decoupling intra-scale interaction and cross-scale fusion, but this will lose a lot of positive local information. To this end, we have improved the efficient hybrid encoder. We propose a new positional encoding method that enables the hybrid encoder to more accurately convert the input feature sequence into a high-dimensional representation, and propose a new feature fusion module to enhance the model's ability to capture local features. Furthermore, in order to improve the tolerance of small objects to IoU, we combine Normalized Wasserstein Distance (NWD) with Shape-IoU for the optimization model. This method more accurately takes into account the shape and size of objects, thereby improving detection accuracy. Our model achieves an accuracy of 38.8% (in terms of mAP<inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">@0.5</inf>) on the widely used VisDrone dataset, which improves the accuracy by 2.5% compared to RT-DETR with ResNet-18 as the backbone network.