Boosting hyperspectral image classification with Gate-Shift-Fuse mechanisms in a novel CNN-Transformer approach

Hyperspectral Image (HSI) classification categorizes every pixel sample into distinct land-cover types. While CNN-based methods have demonstrated significant advances in feature representation, they often struggle with extracting deep features. On the other hand, transformer models excel in capturing high-level semantic features, offering complementary strengths. This paper introduces a novel HSI classification framework that combines convolutional blocks with Gate-Shift-Fuse (GSF) and transformer blocks to leverage CNNs for local feature extraction and transformers for long-range context modelling. The GSF block enhances the integration of spatial–spectral features, while an effective attention mechanism module refines feature extraction from HSI cubes. The proposed method was evaluated on four benchmark datasets (Indian Pines, Pavia University, WHU-Hi-LongKou, and WHU-Hi-HanChuan). It achieved state-of-the-art overall accuracies (OA) of 99.47% on the Indian Pines dataset and 99.86% on WHU-Hi-LongKou, demonstrating significant improvements over existing models. These results highlight the framework’s ability to address challenges like spectral variability and unbalanced datasets. Our contributions include the integration of the GSF block for enhanced spatial–spectral feature fusion and the application of transformers for robust high-level semantic understanding, setting a new benchmark in HSI classification. • We introduced the Gate-Shift-Fuse (GSF) designed specifically for HSI classification. • GSF strengthen CNNs and transformers in extracting local and global features. • A local-global attention mechanism module for HSI cubes is proposed. • long-range context modelling leveraged by strengths of CNNs and transformers. • Superior classification performances achieved on well known HSI benchmark datasets.