Multi-Scale Adaptive Diffusion Feature Fusion with Causal Invariance Learning for Hyperspectral Image Classification

Hyperspectral image (HSI) classification is a significant research area in remote sensing with a wide range of application scenarios. Recently, numerous HSI classification methods based on convolutional neural networks (CNNs) and Transformers have demonstrated promising classification performance. However, these methods demonstrate insufficient capability in mining spectral–spatial relationships from limited HSI samples and fail to extract features pertinent to the target category. To address these challenges, we propose a multi-scale adaptive diffusion feature fusion and causal invariance learning (MSDF-CIL) framework based on diffusion models. Specifically, the framework establishes spectral–spatial distribution relationships through forward and backward diffusion processes. The forward process gradually introduces noise to the HSI input. In the backward diffusion process, our pre-trained hyperspectral denoising network extracts semantically rich multi-scale diffusion features from complex spectral–spatial relationships. A multi-scale adaptive diffusion feature fusion (MSADFF) module is designed to learn key information about each scale and fuse it to enhance the representation. In addition, a causal invariance learning (CIL) module is designed to focus on features causally related to the target class, enabling the model to eliminate spurious correlations among diffusion features. Experimental results on three public HSI datasets show that the proposed MSDF-CIL outperforms other state-of-the-art HSI classification methods, even with minimal samples. When the number of training samples in each class reaches 30, the MSDF-CIL achieves overall accuracy improvements of 4.0% on the Pavia University dataset, 2.3% on the Indian Pines dataset, and 2.3% on the Houston13 dataset, respectively.