Spatial-Spectral Transformer for Local and Global Hyperspectral Image Classification

Deep learning techniques have become increasingly prevalent in hyperspectral image (HSI) classification. However, convolutional neural network (CNN)-based approaches often struggle with inefficient utilization of spectral sequence data and inadequate processing of fine-grained spectral details. In contrast, transformer models are inherently designed to capture sequence dependencies across spectral bands. Yet, most existing approaches tend to focus on either local or global feature extraction separately, which limits their ability to fully exploit the spectral information. To overcome these limitations, we propose the Local and Global Fusion Spectral Transformer (LGFST), an innovative architecture that combines Swin Transformer blocks, standard Transformer blocks, and a novel Local-Global Fusion (LGF) module. This architecture is designed to simultaneously capture both local and global features, thereby improving the extraction of detailed spectral information. The Swin Transformer block, integrated with patch merging and hierarchical attention mechanisms, expands the receptive field and enhances local feature representation through both intra- and cross-window interactions. Meanwhile, the Transformer block performs self-attention across all tokens to extract global features. The fusion of local and global features from these two blocks effectively improves the texture and discriminative power of spectral data, leading to enhanced classification performance, especially in scenarios with dense class distributions. Finally, the LGF module facilitates interaction between local and global feature sets, ensuring a seamless integration of information from both blocks. Extensive experiments on three benchmark HSI datasets demonstrate that LGFST outperforms conventional transformer models and sets new benchmarks in performance compared to state-of-the-art methods.