SpectralDINO: Dual Mixture-of-Subspaces Low-Rank Adaptation for Cross-Domain Hyperspectral Image Few-Shot Classification

Recently, few-shot learning-based methods have achieved impressive results in cross-domain hyperspectral classification. However, existing approaches often ignore differences in spectral information caused by varing spectral range across different datasets, and encounter limitations due to the constrained parameter size of the models. Furthermore, the substantial differences between RGB and hyperspectral images present significant challenges in applying foundation models (e.g., SAM, DINOv2) to the hyperspectral domain. This paper proposes a novel framework for cross-domain few-shot hyperspectral classification that leverages parameter-efficient fine-tuning, which we apply to DINOv2 to construct SpectralDINO. Different spectral ranges reflect different physical properties of the target. To enhance the consistency of the spectral features extracted by the model from different domains, we introduce a source domain spectral alignment (SDSA) strategy to align the spectral ranges and bands of the source domain data to the target domain. SpectralDINO employs the visual foundation model to enhance its ability to generalize cross-domain knowledge. Additionally, we propose a dual mixture-of-subspaces low-rank adaptation (Dual-MoS LoRA) method to address the structural limitation of the low-rank adaptation methods in distinguishing domain-specific features from multi-domain inputs. Only 1.14% of the 21.37M parameters need to be trained to perform fine-tuning. Extensive experimental results on three public datasets demonstrate the superiority of SpectralDINO.