Adaptive Conformal Prediction Intervals for Invariant Learning

Distribution shifts between training and testing datasets can significantly affect the effectiveness of machine learning models, as the typical assumption of data homogeneity often does not hold in practical applications. To address this challenge, previous literature explores invariant representation learning methods, including but not limited to domain adversarial learning, feature alignment, and invariant risk minimization. In this paper, we introduce a novel method for generating distribution-free prediction regions, which effectively quantify the uncertainty of predictions in different environments. Our methodology incorporates a weighted conformity score, tailored to the environment of each test sample, to construct adaptive conformal intervals. The proposed method is computationally efficient and can be implemented by subsampling. We demonstrate how these intervals maintain a consistent conditional coverage under some conditions. The numerical experiments, including simulations and real data analysis, show that our method is applicable and robust in dealing with distribution shifts.