Spatial contextual learning for hyperspectral imagery classification using transformer encoder framework based ensemble model

Abstract Hyperspectral image (HSI) classification remains a challenging problem in remote sensing. This is mainly due to the high dimensionality of spectral data, the limited availability of labeled samples, and the need to jointly model spectral and spatial information. Recent studies have shown that Transformer encoders are effective for spectral modeling. Convolutional architectures are also widely used for spatial feature extraction. However, their integration remains relatively underexplored, particularly under label–scarce conditions. In this work, we propose a four–stage hybrid framework for hyperspectral image classification. The framework integrates spectral modeling and spatial feature extraction within a unified pipeline. First, a multiscale 1D convolutional module performs spectral reduction using parallel kernels of sizes 3, 7, and 11. These kernels capture spectral patterns at multiple resolutions. The module compresses the original spectral bands into a 32–dimensional embedding while preserving important spectral information through a residual skip connection. Second, a pixel–wise Transformer encoder models long–range spectral dependencies in the compressed token sequence. The encoder uses four-head self-attention to capture non–local spectral relationships. Third, a U-Net based spatial extractor captures spatial context within the hyperspectral scene. Hierarchical encoder–decoder skip connections help preserve spatial information and recover fine–grained boundary details. Finally, a dual–stream ensemble classification head aggregates complementary feature descriptors. It combines global average pooling and global max pooling to improve prediction robustness. We evaluate the proposed architecture on two benchmark datasets: Indian Pines and Pavia University. Experiments are conducted under both standard training conditions (80%) and limited–label settings (5%, 10%, and 15%). We also compare the model with recent baselines, including DaViT. On the Indian Pines dataset with 80% training data, the model achieves 99.85% overall accuracy and a weighted F1 score of 99.53%. The Cohen’s $$\kappa$$ value reaches 0.9945 while the model uses only 7.7 M parameters.