Text-Assisted Multimodal Adaptive Registration and Fusion Classification Network

Due to inherent sensor discrepancies, hyperspectral image (HSI) and LiDAR data in fusion classification tasks often suffer from spatial misalignment and limited high-quality annotations, which severely constrain classification performance. While some recent methods attempt to address these issues by introducing semantic alignment strategies or contrastive learning (CL), challenges such as inconsistent cross-modal representations and limited semantic generalization still persist. To address these issues, we propose a novel text-assisted adaptive registration and fusion classification network (TARCNet). First, we develop a feature adaptive alignment (FAA) module, which adaptively adjusts LiDAR features to alleviate semantic inconsistency under misregistration. Second, we introduce a text-assisted contrastive learning (TCL) module, which leverages linguistic priors to strengthen cross-modal consistency and improve the discriminability of the learned representations. Third, we incorporate a multi-loss joint optimization (MLO) module to ensure consistent and stable optimization across heterogeneous modalities. Extensive experiments conducted on three HSI-LiDAR datasets with misregistration and limited annotations demonstrate that our method outperforms several state-of-the-art approaches, validating its effectiveness and generalization capability.