RS-DeCLIP: Gated Dual-Source DeCLIP Adaptation with Side-Branch Query-Based Local Fusion for Remote Sensing Image–Text Retrieval

Remote sensing image–text retrieval requires diverse textual supervision and fine-grained visual representations. Redundant captions limit linguistic diversity, while rewriting can introduce semantic drift. A single global image descriptor may also underrepresent small objects and local spatial arrangements, limiting fine-grained discrimination. To address these challenges, we propose RS-DeCLIP, a two-stage retrieval framework. Stage 1 adapts a RemoteCLIP ViT-B/32 dual encoder with DeCLIP-style multi-view supervision using official captions and captions polished by a large language model. A semantic consistency gate downweights text self-supervision (TSS) between less consistent original and rewritten captions. Similarity-aware negative downweighting and text-to-image weighting address ambiguous negatives and directional imbalance, while training-only fine-grained alignment strengthens detail sensitivity. Stage 2 targets the fine-grained visual information underrepresented by the global descriptor. Five learnable visual query vectors, conditioned on the global image embedding and guided by soft spatial preferences, attend to a dense P2 feature map to extract local evidence. Consistency-gated orthogonal residual fusion integrates this evidence into the frozen global image embedding. These visual queries are distinct from the user’s text query: image feature extraction and fusion do not use the retrieval text, so fused image descriptors can be precomputed and compared with independently encoded text vectors. On the Remote Sensing Image Captioning Dataset (RSICD), Stage 1 improves mean recall (MR) from 35.26% to 38.54%, and Stage 2 further increases MR to 39.27%. On the Remote Sensing Image–Text Match Dataset (RSITMD), Stage 2 improves MR from 50.90% to 52.58%. The final model exceeds the reported GeoRSCLIP-FT (RET-2) results by 0.40 and 0.77 percentage points on RSICD and RSITMD, respectively, and the Representation Discrepancy Bridging (RDB) method by 1.18 and 0.54 percentage points.

Authors

Institutions

Publication Details

Journal
Remote Sensing
Published
2026-11-19
DOI
https://doi.org/10.3390/rs18223986
Primary Topic
Advanced Image and Video Retrieval Techniques
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

RS-DeCLIP: Gated Dual-Source DeCLIP Adaptation with Side-Branch Query-Based Local Fusion for Remote Sensing Image–Text Retrieval

Yun Liao, Junhui Liu, Qi Cui, Qing Duan et al.
Remote Sensing
Advanced Image and Video Retrieval Techniques
article

RS-DeCLIP: Gated Dual-Source DeCLIP Adaptation with Side-Branch Query-Based Local Fusion for Remote Sensing Image–Text Retrieval

Yun Liao, Junhui Liu, Qi Cui, Qing Duan, Chang Yang, Cheng Liao
article en

Abstract

Remote sensing image–text retrieval requires diverse textual supervision and fine-grained visual representations. Redundant captions limit linguistic diversity, while rewriting can introduce semantic drift. A single global image descriptor may also underrepresent small objects and local spatial arrangements, limiting fine-grained discrimination. To address these challenges, we propose RS-DeCLIP, a two-stage retrieval framework. Stage 1 adapts a RemoteCLIP ViT-B/32 dual encoder with DeCLIP-style multi-view supervision using official captions and captions polished by a large language model. A semantic consistency gate downweights text self-supervision (TSS) between less consistent original and rewritten captions. Similarity-aware negative downweighting and text-to-image weighting address ambiguous negatives and directional imbalance, while training-only fine-grained alignment strengthens detail sensitivity. Stage 2 targets the fine-grained visual information underrepresented by the global descriptor. Five learnable visual query vectors, conditioned on the global image embedding and guided by soft spatial preferences, attend to a dense P2 feature map to extract local evidence. Consistency-gated orthogonal residual fusion integrates this evidence into the frozen global image embedding. These visual queries are distinct from the user’s text query: image feature extraction and fusion do not use the retrieval text, so fused image descriptors can be precomputed and compared with independently encoded text vectors. On the Remote Sensing Image Captioning Dataset (RSICD), Stage 1 improves mean recall (MR) from 35.26% to 38.54%, and Stage 2 further increases MR to 39.27%. On the Remote Sensing Image–Text Match Dataset (RSITMD), Stage 2 improves MR from 50.90% to 52.58%. The final model exceeds the reported GeoRSCLIP-FT (RET-2) results by 0.40 and 0.77 percentage points on RSICD and RSITMD, respectively, and the Representation Discrepancy Bridging (RDB) method by 1.18 and 0.54 percentage points.

Remote SensingVol. 18(22)
Yunnan University (CN)
Advanced Image and Video Retrieval Techniques
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon