RetinaNet-XAG: Multicategory SAR Target Detection Under Sample Imbalance With Oriented Bounding Boxes

Fine-grained classification of multi-category vehicles and ships in synthetic aperture radar (SAR) imagery and accurate orientation estimation remains challenging. This is mainly due to the imbalance among different classes, the lack of hard examples, and the presence of complex background clutter. Existing methods are often limited to single-class detection and demonstrate inadequate robustness in open-world scenarios. To address these limitations, a novel multi-category SAR object detection framework, RetinaNet-XAG, is proposed under multi-category sample-imbalanced conditions with oriented bounding boxes. The architecture incorporates a ConvNeXt backbone to enhance multi-scale feature representation and suppress background interference. An improved Adaptive Training Sample Selection strategy for Oriented bounding boxes (ATSSO) is introduced to balance positive and negative sample selection. Additionally, the Gaussian Wasserstein Distance (GWD) loss is adopted to improve regression accuracy for rotated bounding boxes. A Redundant Oriented Bounding Box Removal (ROR) module is further designed to eliminate conflicting multi-category predictions for single targets. To validate the proposed method under sample imbalance, a new benchmark dataset named the SAR-based Diverse Vehicle and Ship Dataset (SDVSD) is constructed, containing 19 categories sourced from multiple SAR platforms. Experimental results demonstrate that RetinaNet-XAG outperforms state-of-the-art methods in mean average precision (mAP), recall, and F1-score metrics, while exhibiting strong generalization capabilities in large-scale SAR application scenarios.