MRC-Net: A Multistage Network With Rank-Reduced Multihead Self-Attention and Cascade Learning for Hyperspectral Pansharpening

Hyperspectral (HS) pansharpening aims to enhance the spatial structure of Low-Resolution (LR) HS images guided by High-Resolution (HR) Panchromatic (PAN) images while maintaining spectral fidelity of resulting HR-HS images. Even though single-stage architecture dominates current state-of-the-art approaches and delivers competitive performance, they may inadequately capture the intricate spatio-spectral dependencies in HR-HS images in only a single stage. In addition, the redundant information of HS images incurs heavy computational overhead. To this end, we propose a Multi-stage Network with Rank-reduced multi-head self-attention and Cascade learning (MRC-Net), which progressively reconstructs the ideal HR-HS images. Our method gradually incorporates high-frequency details from the PAN image into the LR-HS images through multiple stages, enabling the comprehensive learning and processing of the interactive relationship between high-dimensional spectral and complex spatial structures. Additionally, we introduce a rank-reduced multi-head self-attention module at substage levels. As such, redundant information of HS images can be reduced so as to alleviate computational burden, and the learning capacity of MRC-Net is improved to capture long-range dependencies and enhance global contextual relevance. To further obtain the high-dimensional spatio-spectral relationship, we optimise the MRC-Net using a cascade loss function with adaptive weighting across all stages, which guides the learning process and improves the overall performance. Extensive evaluations on three publicly available datasets (Pavia Centre, Chikusei, and Houston 2018) demonstrate that the proposed MRC-Net outperforms the state-of-the-art pansharpening methods in both quantitative comparison and qualitative analysis, which highlights its effectiveness in both spatial enhancement and spectral preservation.