When Feature Alignment Helps, Hurts, or Washes out: Carrier-Dependent Domain Adaptation for Cherenkov Telescope Event Classification

Feature alignment domain adaptation (DA) methods such as CORAL and MMD are widely used to mitigate distribution shift between training (source) and deployment (target) domains. They are typically evaluated on a single downstream classifier, which implicitly assumes that the alignment benefit transfers across model families. We test this assumption on a controlled class-symmetric shift of Imaging Atmospheric Cherenkov Telescope features, with five carrier classifiers, three alignment methods, and fifteen seeds per cell. Establishing this requires first correcting an evaluation protocol that we believe to be common in benchmarks built by perturbing a single sample: when the target consist of a perturbed copy of data that the carrier was partly trained on, the measured benefit of adaptation is inflated by a factor that tracks the carrier’s capacity to memorise (from 1.0 for logistic regression to 5.7 for an unbounded-depth random forest), so that such a benchmark reports a carrier gradient whether or not one actually exists. When scored only on events that no carrier has seen, the carrier still matters (indeed, twice over): the same shift costs a hyperplane carrier 0.004 of AUC and a tree ensemble 0.006–0.008, and CORAL then recovers 58% of that loss on logistic regression, against 84–98% on the tree ensembles, a difference that RBF-kernel MMD does not show on any carrier for which the ratio is determined. A measurement of marginal sensitivity taken without any adaptation reproduces the ordering. Two effects that would be reported by a full-target protocol are absent on held-out events: a benefit growing with tree depth, and an advantage of covariance alignment over mean matching on deep trees. We propose a two-axis framework (shift symmetry × carrier marginal sensitivity), then test its class-symmetric row and give practical recommendations for evaluating and deploying domain adaptation under shift.