SSHFormer: Optimizing Spectral Reconstruction with a Spatial–Spectral Hybrid Transformer

Reconstructing hyperspectral images (HSIs) from RGB images is an effective technique to overcome the high cost of spectrometers. Recently, Transformers have shown potential in capturing long-range dependencies for spectral reconstruction. However, few Transformer models attempt to simultaneously capture both spatial and spectral correlations in HSIs. Within this study, we introduce an integrated spatial–spectral hybrid Transformer (SSHFormer) framework designed to capture the interplay between spatial and spectral features in HSIs, with the aim of incrementally enhancing the fidelity of the reconstructed HSIs. In SSHFormer, we propose a spatial–spectral multi-head self-attention (SSMA) mechanism, which utilizes dilated convolution to extract non-local spatial features while maintaining parameter efficiency and applies the attention mechanism to the channel dimension to model inter-spectral correlations. Additionally, a 3D feedforward network (3DFFN) is proposed for SSHFormer, which leverages 3D convolution to fuse the spatial and spectral information, enabling more comprehensive feature extraction. Experimental results demonstrate that our SSHFormer achieves state-of-the-art (SOTA) performance on public datasets.