SwinT-SpecNet: hyperspectral image super-resolution reconstruction via dual-branch collaboration and three-stage dynamic fusion

To address the ill-posed problem in hyperspectral image super-resolution (HSISR), existing methods typically introduce prior information of hyperspectral images as regularization constraints. Model-based approaches rely on handcrafted priors, which struggle to comprehensively characterize the complex properties of hyperspectral data. In contrast, learning-based approaches often employ convolutional neural networks (CNNs) to learn implicit priors, but their performance remains limited due to CNNs' focus on local spatial features and insufficient capacity to model long-range dependencies. To overcome these limitations, this paper proposes a novel dual-branch cross-attention Transformer-CNN network, termed SwinT-SpecNet. The proposed method adopts a dual-branch architecture to separately extract local-global spatial features from multispectral images (MSI) and long-range spectral features from low-resolution hyperspectral images (LR-HSI). The spatial branch integrates Swin-UNet with a shifted window Transformer to enhance the retention of high-frequency spatial details, while the spectral branch combines 1D convolution with spectral self-attention to accurately model nonlinear inter-band relationships. Furthermore, a three-stage feature fusion module—Alignment, Interaction, and Enhancement—is designed. This module aligns cross-modal features using deformable convolution and spectral contrast loss, dynamically associates spatial and spectral features through cross-attention, and enhances feature representation via a hybrid Transformer-CNN structure. Experimental results on the CAVE and Harvard datasets demonstrate that the proposed method improves PSNR by an average of 1.8dB and reduces the spectral angle mapper (SAM) to 3.2° in 4 × super-resolution tasks, significantly enhancing the balance between spatial detail and spectral fidelity compared to state-of-the-art methods (e.g., 3DT-Net, FusFormer). This study provides an efficient multimodal feature co-optimization framework for the HSISR task.