HSD2Former: Hybrid-Scale Dual-Domain Transformer with Crisscrossed Interaction for Hyperspectral Image Classification

An unescapable trend of hyperspectral image (HSI) has been toward classification with high accuracy and splendid performance. In recent years, Transformers have made remarkable progress in the HSI classification task. However, Transformer-based methods still encounter two main challenges. First, they concentrate on extracting spectral information and are incapable of using spatial information to a great extent. Second, they lack the utilization of multiscale features and do not sufficiently combine the advantages of the Transformer’s global feature extraction and multiscale feature extraction. To tackle these challenges, this article proposes a new solution named the hybrid-scale dual-domain Transformer with crisscrossed interaction (HSD2Former) for HSI classification. HSD2Former consists of three functional modules: dual-dimension multiscale convolutional embedding (D2MSCE), mixed domainFormer (MDFormer), and pyramid scale fusion block (PSFB). D2MSCE supersedes conventional patch embedding to generate spectral and spatial tokens at different scales, effectively enriching the diversity of spectral-spatial features. MDFormer is designed to facilitate self-enhancement and information interaction between the spectral domain and spatial domain, alleviating the heterogeneity of the spatial domain and spectral domain. PSFB introduces a straightforward fusion manner to achieve advanced semantic information for classification. Extensive experiments conducted on four datasets demonstrate the robustness and significance of HSD2Former. The classification evaluation indicators of OA, AA, and Kappa on four datasets almost exceed 98%, reaching state-of-the-art performance.