Sparse Information Perception Network for Remote Sensing Cross-Modal Retrieval

In recent years, research on remote sensing (RS) cross-modal retrieval (RSCMR) has gained significant attention, yet the inherent sparsity of RS images has posed challenges in aligning them with corresponding text, leading to reduced retrieval performance. To address this, we propose the sparse information perception network (SIPN), an instance-level dynamic network that adapts to sparse information in RS images and text. Our approach incorporates a dynamic feature fusion module (DFFM) for initializing the network, dynamic training adjustments using random processes and pretrained policy networks, and feature alignment via contrastive loss. Additionally, we enhance feature selection and fusion capabilities through a multilevel attention module (MLAM), which includes cross-attention (CA) and a multilayer perceptron (MLP), with triplet loss for further alignment. Our method demonstrates superior performance in RSCMR on the RSICD and RSITMD datasets.