Domain-Adaptive Deep Transfer Learning for Cross-Modal Heterogeneous Image Matching under Label-Scarce Constraints

Deep learning has made significant strides in cross-modal heterogeneous remote sensing image matching, yet it remains heavily dependent on annotated target scene data for training models. Under label-scarce constraints, domain discrepancies between actual test data and training data often result in a decline in the performance of trained models. To tackle this challenge, a domain adaptation method is put forth for consideration in the paper for cross-modal heterogeneous image matching. The method achieves the transformation between the images of the existing scene and the target scene through style transfer, and narrows the domain offset between the images of the target scene generated by the transformation of images of the existing scene and the original target scene through adversarial learning. It aligns the target domain with unlabeled information and the source domain with labeled information from both the image and feature perspectives, thereby solving the matching tasks inside the target domain through the application of knowledge from the source domain. Experiments were conducted using two different optical and synthetic aperture radar (SAR) cross-modal datasets, alternating as both target and source domains in training and testing. Results confirmed that compared to current traditional matching methods and deep learning matching methods, our method demonstrates feasibility and effectiveness in cross-dataset transfer, effectively reducing the dependency on annotated data for matching tasks in the target scene.