A multiscale transformer with spatial attention for hyperspectral image classification

Hyperspectral images (HSIs) are renowned for their rich spatial and spectral information, which is crucial for accurate classification. The acquisition of discriminative spectral-spatial features plays a pivotal role in determining classification results. While convolutional neural networks (CNNs) have demonstrated remarkable performance in HSI classification, increasing network depth can lead to performance degradation. Furthermore, their fixed scale and limited receptive field restrict the ability to capture long-range dependencies, hindering effective feature learning and, consequently, affecting the generalization capability of the framework. This paper presents a novel HSIs classification framework, MTSA-Net, which integrates a multiscale transformer with a spatial attention mechanism, resulting in a more robust, flexible, and high-performing approach. Initially, the proposed framework utilizes 3-D and 2-D convolution layers, followed by spatial attention to prioritize and focus on the most critical spatial features. These enhanced features are then passed through multiscale transformer encoders to capture local and global representations, effectively modeling long-range dependencies. Finally, a feature fusion module combines features extracted at varying scales, leading to a more robust and comprehensive feature representation for final classification. Extensive experiments on five widely used benchmark HSIs datasets demonstrate that the proposed MTSA-Net method outperforms state-of-the-art approaches, particularly with limited training samples. The overall accuracies of 98.84%, 98.77%, 99.80%, 97.84%, and 95.87% are achieved on the Indian Pines, Pavia University, Salinas Valley, Houston-13, and Houston-18 datasets, respectively. The source code for this work will be accessible at https://github.com/irfan01000 for reproducibility.