IHM-SNet: An Interactive Hierarchical Mamba-Based Screening Network for Remote Sensing Image Change Captioning

Compared with natural images, Remote Sensing (RS) images have multi-spectral and multi-scale characteristics and contain a large amount of complex spatial and temporal information. However, Remote Sensing Image Change Captioning (RSICC) needs to use the dependencies between this temporal and spatial information to generate sentences describing the bi-temporal changes of RS images, this process requires high classification accuracy. Among existing methods, CNN-based and Transformer-based methods have limitations in establishing global receptive fields and their own computational complexity, respectively. The above situation poses challenges to the related work in the field of RSICC. Recently, the popular Mamba model is based on the SSM mechanism and superior hardware algorithm, providing an opportunity to solve the above problems. In this paper, we propose an Interactive Hierarchical Mamba-Based Screening Network (IHM-SNet). The network gradually captures the changes of interest in multi-level bitemporal features in an interactively guided manner through multiple Interactive Hierarchical Encoder (IHE) layers. The Cross Fusion Screening Network (CFS) further to select the key features from the multi-level change features. Experiments verify the effectiveness of our proposed IHM-SNet model for improving the quality of description sentence generation, which provides a valuable exploration for applying Mamba to RSICC in the future.