SSFMamba: Spatial-Spectral Fusion State Space Model for Pansharpening

Pansharpening aims to fuse the panchromatic (PAN) and low-resolution multispectral (LR-MS) images, finally generating the high-resolution multispectral (HR-MS) images by reconstructing the spatial-spectral properties. Recently, VMamba-based methods built upon the visual state space (VSS) have shown great potential in pansharpening, but they are unable to explicitly characterize the spatial-spectral properties from the given LR-MS and PAN images. Thus, we propose spatial-spectral fusion state space model (SSFMamba), which consists of multi-scale spatial-wise visual state space (MSpa-VSS) block, bi-directional spectral-wise visual state space (BSpe-VSS) block, and gated spatial-spectral fusion (GSSF) block. Specifically, the MSpa-VSS block introduce multi-scale convolution opeartion in the VSS, thus simultaneously modelling spatial dependencies and capturing multi-scale spatial property; the BSpe-VSS block bi-directionally scans the spectral channels for learning continuous spectral property; moreover, we design the GSSF block for fusing spatial-spectral properties in the adaptive manner. Extensive experimental results on several datasets demonstrate the superiority performance of the proposed SSFMamba.