MGWT: Hyperspectral Image Classification Based on a Multi-Granularity Weight Transformer Network

Transformer-based methods have achieved remarkable performance in hyperspectral image (HSI) classification. However, their ability to learn discriminative multi-scale spatial–spectral features remains limited by the single-granularity token-to-token interactions in conventional self-attention and the imbalanced modeling of spatial and spectral information. To address these limitations, we propose a multi-granularity weight transformer (MGWT) network comprising a spatial–spectral grouped parallel residual tokenizer (S2-GPRT) and a multi-granularity attention (MGA) module. Specifically, the S2-GPRT partitions the spectral bands into groups according to their physical characteristics and employs a dual-branch architecture to extract complementary local spatial–spectral features from individual spectral groups and the full spectrum. MGA integrates three components: a multi-granularity aggregator (Agg), a spatial–spectral dual-stream attention mechanism (S2-DSA), and a learnable mixed top-k attention mechanism (LMTA). The Agg preserves a token-level pathway while constructing group-proxy pathways with progressively larger local receptive fields, enabling attention to integrate fine-grained information and multi-scale context within a single block. The S2-DSA captures spatial and spectral dependencies through dedicated spatial and spectral attention streams. Furthermore, the LMTA constructs four rank-guided attention branches with different top-k selection ratios and adaptively fuses their outputs to integrate complementary attention responses at varying levels of sparsity. Experiments on three benchmark datasets demonstrate that the MGWT outperforms state-of-the-art methods in HSI classification.