Hyperspectral Pansharpening with Transformer-Based Spectral Diffusion Priors

Hyperspectral pansharpening aims to fuse a spatially high-resolution panchromatic image (HR-PCI) with a low-resolution hyperspectral image (LR-HSI) to generate a high-resolution hyperspectral image (HR-HSI). Latest deep learning-based methods have achieved notable success in addressing this task, but they often rely on supervised or pre-trained models with high-quality labeled datasets, leading to limitations in practical applications. This paper introduces an unsupervised model that leverages transformer-based diffusion and spectral priors (uTDSP). In this model, spectral priors are learned from LR-HSI images, for which we assume that the stochastic distribution of spectral profiles in an LR-HSI is similar to that in the target HR-HSI. The learned prior is then used to optimize the fusion process by incorporating it as a regularization term, which is estimated simultaneously to adjust the contribution of the diffusion and the learned priors in reconstructing the target HR-HSI. Experimental results on benchmark datasets highlight the proposed method outperforms the state-of-the-art methods. Developed code will be available at this repository.