On Contrastive Learning for Domain Generalization

Given the training data and labels from several seen domains, Domain Generalization (DG) aims to learn models that generalize well on unlabeled data from unseen domains. Due to the distribution of data and/or labels may vary between domains, current DG methods are mainly based on the theme of domain-invariant representation learning which decomposes the learning process into two steps: (1) finding a representation function from the input space to the representation/feature space to learn the so-called domain-invariant features, i.e., the features that are statistically stable between domains, and (2) designing a classifier on top of these domain-invariant features which is mutually optimal for all domains. Recent works on DG show that Contrastive Learning (CL) which was originally proposed to learn a representation function such that similar samples are pulled closer while dissimilar samples are pushed far away in the representation/feature space, appears as a promising solution for domain-invariant features learning. In this paper, we first establish the fundamental theory to justify why CL might be a potential approach for DG by showing that CL, under particular settings, allows a mechanism to minimize a loss function that enforces an optimal classifier for all training domains. Based on these foundations, we revisit the recent works that apply CL for DG to indicate their limitations and suggest a modification by combining the CL method with current DG methods for better out-of-domain generalization. Our numerical results point out that our proposed algorithm can achieve state-of-the-art performance on multiple DG benchmark datasets.