Cross-source transformer-based neighborhood contrastive learning for joint classification of hyperspectral and LiDAR Data

The fusion of the hyperspectral image (HSI) and light detection and ranging (LiDAR) data has demonstrated significant potential in the land cover classification task. Although deep learning has shown remarkable success in the joint classification of HSI and LiDAR data, the large amount of unlabeled multisource remote sensing data is not fully utilized. Additionally, effectively integrating HSI and LiDAR data remains a challenging task, and the semantic relationship of neighborhood regions needs to be further exploited. In this paper, we propose a cross-source transformer-based neighborhood contrastive learning model (CTNCLM), which acquires a more discriminative feature representation from unlabeled data through the pre-training stage. Considering the semantic correlation between neighboring image patches, CTMCLM achieves the joint classification of HSI and LiDAR data at a more precise level. A cross-patch contrastive learning (CPCL) module is proposed to calculate the similarity between original patches and neighborhood patches. Furthermore, a cross-source transformer (CST) with cross-source attention is proposed to fuse the multi-source data, which exploits the intermodal information interaction between the HSI and LiDAR data. Extensive experiments on three public datasets demonstrate the superior classification performance of the proposed method compared with several state-of-the-art methods.