HyperESSANet: An Efficient Spatial-Spectral Attention Network for Hyperspectral Pansharpening

Existing remote sensing hyperspectral imaging sensors (HISs) typically capture hyperspectral images (HSIs) with relatively low spatial resolution due to the trade-off between spectral and spatial resolution. As a solution, hyperspectral pansharpening aims to improve the spatial resolution of HISs by fusing HSIs with panchromatic images (PANs). Existing hyperspectral pansharpening methods often face challenges such as inadequate fusion of spectral and spatial information, excessive GPU memory usage, and prolonged inference times. To address these issues, we propose HyperESSANet, an end-to-end network for efficient hyperspectral pansharpening. Specifically, HyperESSANet consists of the gated channel attention block (GCA-Block) encoder, the efficient spatialspectral attention network (ESSA-Net), and the convolutional block (Conv Block) decoder. The GCA-Block encoder introduced a novel gated channel attention module, which allows the encoder to focus better on important spectral features. The ESSA-Net is a novel spatial-spectral attention network with a weight-shared downsampling strategy, which promotes cross-scale fusion of deep spatial-spectral features. The Conv Block decoder is utilized to effectively reconstruct the high-resolution HSI. Furthermore, the ESSA-Net is integrated between the encoder and the decoder, allowing the network to dynamically fuse spatial and spectral information across multiple scales. Experimental results on public HSI datasets show that the proposed HyperESSANet outperforms existing methods in both image quality and memory efficiency. The proposed HyperESSANet is implemented in PyTorch and will be available at https: //github.com/Anywayany/HyperESSAN.