Language-Guided Object Localization via Refined Spotting Enhancement in Remote Sensing Imagery

Language-guided remote sensing image object localization uses intuitive natural language interactions to locate objects of interest within satellite or drone imagery, and has a wide range of practical applications. Early research on this task typically used discriminative models that relied on predefined task heads, limited to locating a single object and lacking flexibility. Recent years, multi-modal large language models based generative models leverage their language understanding ability to comprehend more complex language references and provide flexible outputs. However, these models tend to be too large and have limited accuracy in spotting dense, small objects in remote sensing scenarios. The reasons are: 1) Generative models treating continuous coordinate prediction as a token classification problem which fails to reflect the actual localization gap. 2) Current methods typically use CLIP pre-trained encoders that align text-image pairs at a global level, may not capture the fine-grained semantic object information necessary for accurate localization. To address these shortcomings, we introduce a lightweight generative model named LM-RSE (Localization Model with Refined Spotting Enhancement). Refined Spotting Enhancement includes the refinement of bounding box outputs and the integration of fine-grained semantic features. Specifically, we design a Bounding Box Refinement (BBR) approach that includes a special token, a Bounding Box decoder, and a custom regression loss function to refine the spotting precision and optimize the training process. Additionally, we propose a Fine-Grained Semantic Integration (FGSI) strategy, integrates a fine-grained image encoder, a Vision-Language Semantic Processing (VLSP) layer and a two-stage, full-parameter training strategy, all working together to effectively enhance the granularity of spotting. Building on this, we further refine the language-guided object localization task into two types: expression-guided and class-guided. For the former, we utilize the RSVGD dataset for evaluation and achieved state-of-the-art performance; for the latter, our evaluation results surpassed those of GeoChat. Our code and checkpoints will be released at https://github.com/Zhang-Peirong/LM-RSE.