Energy-Regularised Novelty Scores and Approximate Conditional Coverage for Conformal Prediction under Open-Set Label Shift

Under open-set label shift, a conformal predictor calibrated on the observed classes is applied to a test distribution that also contains unseen classes. A common remedy complements the prediction set over the observed labels with a flag for novel inputs, and practitioners usually derive both from one classifier. We study how to learn the novelty score and what conditional coverage can mean in this setting. First, we add to cross-entropy a logistic loss that discriminates labelled inputs from an unlabelled mixture through the energy of the classifier. At the population level the two losses act on complementary coordinates of the logits, so the minimiser recovers both the posterior and the log density ratio; in our experiments, however, a separate discriminator of matched capacity performs as well. Second, for the observed labels, we specialise rectified conformity scores to a tie-correct adaptive prediction set score, which gives approximate conditional coverage under accuracy assumptions on a learned conditional quantile. Third, we show that conditional coverage over the mixture of observed and novel classes requires flagging every input whose novelty posterior exceeds α, and that this posterior depends on the novel proportion, which is identified only from below. Under a uniform error bound on the learned density ratio and an upper bound on the novel proportion, we construct a flag region that attains the coverage target and bound its excess false-positive rate. An oracle-to-learned ablation shows that, in our simulations, excess flagging is due to error in the learned ratio rather than to partial identification of the proportion.