Feature Interaction and Adaptive Fusion Network With Spectral Modulation for Pansharpening

Pansharpening aims to reconstruct high-resolution 2 multispectral (HRMS) images from low-resolution multispectral 3 (LRMS) images and high-resolution panchromatic (PAN) data. 4 In recent years, deep learning (DL) development has spurred 5 the evolution of data-driven pansharpening methods, which have 6 demonstrated superior performance compared to traditional 7 approaches. However, most existing methods face challenges in 8 preserving spectral information, struggle to effectively interact 9 and aggregate MS and PAN image data, and fail to adaptively 10 coordinate the local textures of high-resolution PAN with the 11 global characteristics of low-resolution MS, leading to suboptimal 12 spectral and spatial fidelity in the generated HRMS images. 13 This paper introduces a feature interaction and adaptive fusion 14 network (FIAFN) with spectral modulation for pansharpening to 15 address these issues. The FIAFN is designed to maintain spectral 16 integrity by incorporating spectral modulation while employing 17 feature interaction attention (FIA) modules and a feature adap18 tive selection attention (FASA) module to fuse complementary 19 information effectively. Specifically, FIA integrates a large re20 ceptive field-selective kernel with efficient multi-scale attention 21 (EMA) to extract both global information from MS images and 22 multi-scale detail information from PAN images. Additionally, 23 a residual structure-based self-guided spatial-channel adaptive 24 convolution (SSCAConv) is introduced to accommodate diverse 25 features within FASA adaptively. Comprehensive experiments 26 demonstrate that our proposed FIAFN achieves superior per27 formance compared to recent methods on the QuickBird (QB), 28 WorldView-3 (WV3), and WorldView-2 (WV2) datasets.