Integrating Global and Local Information for Remote Sensing Image–Text Retrieval

Pre-trained Vision-Language Models (VLMs) have demonstrated promising performance in remote sensing image-text retrieval tasks. However, the scarcity of high-quality image-text datasets remains a challenge in fine-tuning VLMs for remote sensing. The captions in existing datasets tend to be uniform and lack details. To fully utilize rich detailed information from remote sensing images, we propose a method to fine-tune VLMs. We first construct a new visual-language dataset that balances both Global and Local information for Remote Sensing image-text retrieval (GLRS). Specifically, a Multi-modal Large Language Model (MLLM) is utilized to generate captions for local patches and global captions for the entire image. To effectively utilize local information, we propose a Global and Local image Captioning method (GLCap). With a Large Language Model (LLM), we further obtain higher-quality captions by merging both global and local captions. Finally, we fine-tune the weights of RS-M-CLIP with a progressive global-local fine-tuning strategy on GLRS. Experimental results demonstrate that our method outperforms state-of-the-art approaches on two common remote sensing image-text retrieval downstream tasks. The dataset will be publicly available once the paper is accepted.