Long-Tailed Remote Sensing Image Classification via Multi-Scale Data, Pre-Trained Model, and Efficient Inference Strategy

Remote sensing image classification is one of the fundamental tasks in the field of remote sensing and plays a critical role in Earth observation applications. However, the inherent multi-scale characteristics of this task pose significant challenges to scene classification. To address these issues, we propose a novel framework that integrates the Contrastive Language–Image Pre-training (CLIP) model, multi-scale data, and efficient inference strategy. The proposed framework transfers general-purpose features learnt from natural images to remote sensing image classification. Specifically, this framework leverages the rich feature representations learnt by the CLIP model in the contrastive learning procedure and adopts it as the backbone network of the model to extract fine-grained and multi-scale features for remote sensing images. That is, the model can learn local fine-grained details but also encode global contextual information useful for the classification of visually similar scene categories. Afterwards, AdapterFormer module is inserted into the few selected layers of CLIP model, which can effectively enhance model performance and have low computational overhead. This helps efficient knowledge sharing and introduces new features at the model level. Furthermore, to alleviate possible performance deterioration brought about by multi-scale feature variation, a multi-scale training set is constructed at data level, providing complementary multi-scale information. Through the synergy of all these strategies above, the proposed method greatly improves the classification performance of multi-scale remote sensing images. Extensive experiments on the MEET dataset (it includes 80 fine categories and more than 800,000 samples) show that the proposed method greatly improves the performance. Compared with general-purpose classification networks and remote sensing-related models, the proposed method always gets state-of-the-art results.