Text-guided Unsupervised Domain Adaptation

Unsupervised domain adaptation (UDA) has gained significant attention as a methodological framework to mitigate domain shift between annotated source data and unlabeled target distributions. Recent scholarly investigations have been directed towards harnessing the capabilities of large-scale vision-language models, particularly CLIP, through strategies involving model fine-tuning or refining prompt-based learning paradigms for UDA implementation. Building upon these developments, our study capitalizes on the intrinsic semantic diversity and cross-domain generalization properties inherent in pre-trained CLIP architectures. We establish a novel domain discrepancy metric by quantifying feature alignment between source-target domains, thereby introducing an innovative CLIP-based UDA methodology designated as Text-guided Unsupervised Domain Adaptation (TG-UDA). Specifically, since text can more succinctly capture the semantic essence, TG-UDA integrates textual prompts from the source and target domains to establish a domain-agnostic distribution using CLIP, serving as a bridge between the source and target domain data. Additionally, divergence loss and distance loss are proposed to quantify the differences between domain-specific distributions and the domain-agnostic distribution, guiding the UDA model to align the feature spaces extracted from the source and target domain data, thereby reducing domain divergence. Furthermore, current pseudo-label-based unsupervised domain adaptation methods often utilize only a limited amount of target domain data. To address this limitation, we propose a novel text-prompt-based pseudo-label generation method. Under the influence of a reward mechanism for positive samples and a penalty mechanism for negative samples, this approach significantly improves the utilization efficiency of source domain data, ultimately enhancing the model’s performance on target domain data.