The CNN-Transformers Crossroads, Comparing RT-DETR and YOLOv12 for Small object detection in remote sensing images

Abstract. Detecting small objects in remote sensing images has always been a challenge. The Convolutional Neural Network (CNN) and Transformer-based networks are two prominent categories of deep learning models used to address this challenge. Recently, combining both architectures has emerged to improve detection performance. However, a direct comparison between the leading standard models representing these architectures has yet to be conducted. In this study, we provided a performance comparison of two state-of-the-art detectors: YOLOv12, a CNN-based model with an attention mechanism, and RT-DETR, a transformer-based model built on a CNN backbone. We fine-tuned both algorithms on a custom remote sensing dataset containing small objects (airplanes and cars) and evaluated their performance based on precision, recall, F1-score, and training time. The results showed that YOLOv12 was significantly faster to train and achieved higher precision. These qualities make it a better choice for applications where minimizing false positives is critical. RT-DETR, with high recall and F1-score, was more effective at detecting a larger number of small objects. This analysis offers valuable insights into the trade-offs between these two architectures and serves as a guideline for selecting the appropriate model for each specific remote sensing task.