Frequency-Spatial Domain Information Fusion Network for Pan-Sharpening

Pan-sharpening aims to fuse panchromatic (PAN) images with low-resolution multi-spectral (LR-MS) images to generate high-resolution multi-spectral (HR-MS) images. Despite the impressive performance of existing learning-based methods, they are constrained by coarse fusion strategies in frequency or spatial domain. In this paper, we discover that PAN images can provide all the spatial textures required for HR-MS images, while spectral information must be provided jointly by PAN and LR-MS images. Inspired by this, we propose a noval frequency-spatial domain information fusion network for pan-sharpening, called FSDNet. Specifically, we design a Dual-Domain Information Processing Module (DDPM) to construct FSDNet. It consists of a Frequency Domain Feature Processing Block (FDB), a Spatial Domain Information Processing Block (SDB), and an Information Fusion Block (IFB). The FDB in the frequency domain uses the Adaptive Amplitude Fusion Block (AAFB) and convolution layers to finely modulate amplitude and phase components, exploring global information. The SDB uses cascaded residual blocks to capture and enhance local information in the spatial domain. The IFB based on invertible neural networks (INNs) introduces Multi-Scale Self-Attention Block (MSAB), achieves effective information fusion and reduce information loss. Extensive experiments on the QuickBird and GaoFen-2 datasets demonstrate the effectiveness and superiority of our method.