MCIM: Modal Collaborative Interactive Mamba Network for Hyperspectral and Multispectral Image Fusion

Hyperspectral and multispectral image fusion aims to generate hyperspectral image with high spatial resolution, thereby enhancing their utility in various downstream applications such as classification, detection, and segmentation. However, there are not only differences but also compex correlations between hyperspectral images and multispectral images in both spatial and spectral dimensions. This leads to certain differences and conflicts in the extracted image features, affecting the subsequent fusion results. To overcome these issues, our work designed a modal collaborative interactive Mamba network (MCIM) for hyperspectral and multispectral image fusion. First, we propose a spatial window matching attention module to extract the spatial features that can be matched and fused. At the same time, we propose a spectral sequence matching attention module to extract the spectral features that can be matched and fused. These features represent complementary information in spatial and spectral dimensions respectively. Second, we propose a cross mamba fusion module (CMFM). The extracted features are input into the cross mamba fusion module for feature fusion and reconstruction. CMFM adds cross thinking to the conventional mamba architecture, and generates a joint state space to promote cross modal information interaction, so that it has the effect of cross modal feature fusion. Finally, through multiple feature extraction, these features are input into CMFM after each feature extraction. Use cross mamba to generate joint state space, interactively fuse features, and reconstruction of hyperspectral image with high spatial resolution. Experimental results show that the proposed MCIM is superior to the most advanced method, especially for cross modal information fusion.