Sample-Level Improved Cross-Source Contrastive Learning for PAN and MS Joint Classification

In recent years, the number and ways of acquiring panchromatic images (PAN) and multispectral images (MS) have increased, and manual labeling costs have also increased. Processing these data efficiently has become a challenge. In this paper, we propose a sample-level improved cross-source contrastive learning method for PAN and MS joint classification (SLCL), which aims to provide a self-supervised pre-training model using unlabeled samples for downstream joint classification using a small quantity of labeled samples. First, we propose a sample weighting and screening (SWS) strategy, which enables the model to learn inter- and intra-source sample representations, while balancing the interference from false samples so that the model learns true samples. It solves the problems of homologous similar features embedded far away and false negative samples bringing the wrong learning direction, which exist in existing contrastive learning methods. In addition, we design a hard sample learning (HSL) module for the problem of mining and optimization of hard samples. The module efficiently mines hard samples and uses a new loss function to make the model more focused on hard sample optimization. It further improves the accuracy of pre-training models for downstream tasks. Our method performs best on multiple datasets, and it is experimentally validated and analyzed. The code is available at: https://github.com/Xidian-AIGroup190726/SLCL.