Aero‐ VLM : Vision‐Language Guided Oriented Aircraft Detection for Remote Sensing Imagery

ABSTRACT Fine‐grained aircraft detection in remote sensing (RS) imagery is challenged by limited labelled data, domain shifts, high intra‐class similarity and complex environmental conditions. Conventional detectors rely heavily on dense supervision, whereas vision–language models (VLMs) often lack the geometric precision required for accurate localization in aerial scenes. To address these challenges, we propose Aero‐VLM, a hybrid CNN–Transformer framework that combines self‐supervised visual representation learning with pretrained vision–language semantic embeddings for data‐efficient fine‐grained aircraft detection. An oriented detection head with knowledge distillation is integrated with the vision–language representation to improve geometric localization and support efficient inference. We further introduce Aero‐RSI, a 10,000‐image high‐resolution benchmark comprising ten visually similar aircraft categories for fine‐grained detection. Aero‐VLM achieves 64.6% mAP@50 in zero‐shot evaluation on unseen classes, outperforming CLIP, BLIP‐2, Grounding DINO and VisionLLM by up to 44.1 percentage points. With only 10 labelled samples per class, Aero‐VLM improves mAP@50 by 16.1 percentage points, whereas competing models achieve gains of less than 10 percentage points. Cross‐domain evaluation on DOTA, MTARSI and FAIR1M demonstrates strong generalization without additional fine‐tuning, with Aero‐VLM achieving gains of up to 11.0 percentage points over CNN‐based detectors and up to 46.0 percentage points over adapted VLMs. Under full supervision, Aero‐VLM achieves 90.2% mAP@50, exceeding YOLOv11s by 2.5 percentage points and Grounding DINO by 39.7 percentage points. Despite a 33% increase in inference latency relative to YOLOv11s, Aero‐VLM requires approximately 45% less inference time than Grounding DINO, demonstrating a favourable accuracy–efficiency trade‐off.