A global–local dual-stream collaborative enhancement model for cross-modal image and text retrieval in remote sensing scenarios

Remote sensing image-text cross-modal retrieval technology serves as a key supporting technology for transforming remote sensing big data into knowledge services, and it holds significant application value in national strategic areas such as smart city development and ecological environment protection. To address the limitations of single feature representation caused by the multimodal object types and complex texture spatial structures of remote sensing images, this paper proposes a cross-modal retrieval framework based on global–local feature collaborative enhancement. In terms of visual representation, a local feature branch using graph convolutional networks is introduced based on contrastive language-image pre-training global features, along with a local–global feature synergistic enhancement module designed to achieve self-attention enhancement of multi-scale features, complementary calibration, and hierarchical fusion through an adaptive weight allocation strategy. For text representation, an attention-convolution fusion module is designed to enhance the semantics of the text and align it with visual information. Finally, cosine similarity is employed to compute the semantic correlation between visual and text features, and contrastive loss is used for model optimization. Experiments on the RSITMD and RSICD datasets verify the effectiveness of the proposed method.

Authors

Institutions

Publication Details

Journal
AIP Advances
Published
2025-08-01
DOI
https://doi.org/10.1063/5.0280560
Primary Topic
Multimodal Machine Learning Applications
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

A global–local dual-stream collaborative enhancement model for cross-modal image and text retrieval in remote sensing scenarios

Jingqi Zhang, Huiying Ru, Jiaqi Ji, Juan Wen et al.
AIP Advances
Multimodal Machine Learning Applications
article

A global–local dual-stream collaborative enhancement model for cross-modal image and text retrieval in remote sensing scenarios

Jingqi Zhang, Huiying Ru, Jiaqi Ji, Juan Wen, Yuansheng Xia
article en

Abstract

Remote sensing image-text cross-modal retrieval technology serves as a key supporting technology for transforming remote sensing big data into knowledge services, and it holds significant application value in national strategic areas such as smart city development and ecological environment protection. To address the limitations of single feature representation caused by the multimodal object types and complex texture spatial structures of remote sensing images, this paper proposes a cross-modal retrieval framework based on global–local feature collaborative enhancement. In terms of visual representation, a local feature branch using graph convolutional networks is introduced based on contrastive language-image pre-training global features, along with a local–global feature synergistic enhancement module designed to achieve self-attention enhancement of multi-scale features, complementary calibration, and hierarchical fusion through an adaptive weight allocation strategy. For text representation, an attention-convolution fusion module is designed to enhance the semantics of the text and align it with visual information. Finally, cosine similarity is employed to compute the semantic correlation between visual and text features, and contrastive loss is used for model optimization. Experiments on the RSITMD and RSICD datasets verify the effectiveness of the proposed method.

AIP AdvancesVol. 15(8)
Zhangjiakou Academy of Agricultural Sciences (CN), Hebei University of Architecture (CN), Hebei Normal University of Science and Technology (CN), Hebei Normal University (CN)
Sustainable cities and communities
Openalex Percentile: Top 80%
Multimodal Machine Learning Applications
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.