Adaptive Low-Rank Adaptation Based Parameter-Efficient Tuning for Low-Resource Zero-Shot Remote Sensing Image Classification

Zero-shot remote sensing image classification is a technique that enables models to predict on unseen categories, effectively addressing the challenges associated with the dynamic changes in surface types typical of remote sensing images and the high costs of acquiring extensive labeled data. Currently, shallow neural network algorithms are widely applied in this field However, these algorithms have certain limitations in capturing the complex spatial relationships of land objects. In this study, we transfer the multimodal CLIP model to the remote sensing field, utilizing its advanced training mechanisms to significantly enhance the ability to handle zero-shot tasks in remote sensing image classification by learning the semantic associations between images and text. However, the parameter size of CLIP reaches 151.28M, which brings significant computational costs and limits its practical application in the remote sensing domain. To address these issues, we innovatively introduce AdaLoRA, an efficient parameter fine-tuning strategy designed to reduce the trainable parameter volume and training duration of pre-trained models. This strategy not only maintains the model's classification ability for zero-shot tasks but also makes it more suitable for the specific needs of the remote sensing field. Therefore, in our research, weapply the AdaLoRA strategy to the CLIP model and conduct experiments on the PatternNet, NWPU-RESISC45, and EuroSAT datasets. The results demonstrate that our approach achieves highermodel performance while reducing training time compared to state-of-the-art models.