FDGM: Fourier With Dropout in Global Modeling for Pansharpening

Pansharpening aims to generate high-resolution multispectral (HRMS) images by fusing high-resolution panchromatic (PAN) images with low-resolution multispectral (LRMS) images for various practical tasks such as ground-object classification and change detection. Currently, Transformers have been widely utilized in the pansharpening field because of their capability to model global representations. Transformer-based pansharpening methods capture a global representation of local patch sequences in the spatial domain, but they do not aggregate each pixel in the image, which hinders their ability to capture the global context and dependencies. To solve this issue, a novel pansharpening network based on Fourier with dropout in global modeling (FDGM) is proposed, which can better model global representations and obtain higher-quality fusion images without increasing computational complexity. First, an efficient Fourier with dropout block (FDB) is designed by transforming the features from the spatial domain to the frequency domain to capture the global information from the panchromatic (PAN) and low-resolution multispectral (LRMS) images. This study represents a novel attempt to explore the generalization performance of dropout in the Fourier frequency domain. Furthermore, a residual spatial-frequency fusion module (RSFM) based on frequency channel attention is designed to enhance feature representation capability and effectively facilitate the fusion of the spatial and frequency features. Finally, the shallow features extracted from the source images are combined with the RSFM output to generate the final fusion results. The results of numerous experiments demonstrate that FDGM surpasses other state-of-the-art approaches in visual perception and quantitative evaluation.