Query-Oriented Dynamic Multimodal Alignment for Few-Shot Hyperspectral Image Classification
Deep learning has advanced hyperspectral image classification (HSIC), but label scarcity remains a significant challenge. Traditional unimodal methods usually produce unstable prototypes, while multimodal methods suffer from cross-modal semantic misalignment. Moreover, standard metrics fail to capture high-order feature correlations, further limiting the discriminative ability of the model. To address these issues, we propose a query-oriented dynamic multimodal alignment (QODMA) method, which integrates visual-textual guidance with dual-distance metric learning for few-shot HSIC. Specifically, a query-oriented dynamic attention (QODA) module is designed to bridge the modality gap by aligning visual and textual features through query-driven attention interactions. A bidirectional attention mechanism is constructed to employ contrastive learning to enhance intra-class compactness. Additionally, a dual-distance metric learning (DML) module that combines the Euclidean distance and Brownian distance covariance (BDC) metrics is employed to refine the feature space representation, thereby enhancing the discriminative ability of the model. To mitigate domain shift, an inter-domain structural consistency loss (IDSCL) is constructed. Experimental results on four public hyperspectral data sets demonstrate that the proposed QODMA outperforms state-of-the-art few-shot classification methods.
Authors
- Jiangtao Peng (ORCID: https://orcid.org/0000-0002-4759-0584)
- Weiwei Sun (ORCID: https://orcid.org/0000-0003-3399-7858)
- Qiaoli Zhang
Institutions
- Ningbo University (CN)
- Hubei University (CN)
Publication Details
- Journal
- IEEE Transactions on Geoscience and Remote Sensing
- Published
- 2025-01-01
- DOI
- https://doi.org/10.1109/tgrs.2025.3615225
- Primary Topic
- Remote-Sensing Image Classification
- Type
- article
- Field-Weighted Citation Impact
- 0.00