Multi-modal Remote Sensing Visual Question Answering Algorithm Based on Large Language Model

The field of remote sensing image applications is experiencing unprecedented attention, particularly with the emerging rise of visual question answering (VQA) tasks for remote sensing images. However, constrained by the scale and diversity of existing datasets, traditional approaches often rely on small-scale models, limiting the depth and breadth of understanding. In light of the remarkable advancements in large language models and their vast repository of image and semantic knowledge, we innovatively proposes an Adversarial MultiModal Remote Sensing Image Visual Question Answering algorithm based on Large Language Model(MMRS). It aims to leverage the profound understanding of natural world images and semantic information from large models, transferring their advantages to the domain of remote sensing images. To further enhance the alignment accuracy between image and text modalities, a discriminator module is incorporated into the MMRS algorithm. Moreover, to compensate for the deficiencies in diversity and scale of existing datasets, we constructs a more diverse remote sensing image-text dataset NWPU-GPT. Experimental results demonstrate that the algorithm exhibits significant performance advantages in tests on multiple classic remote sensing datasets.