Watching the Hull: Mixture-Aware Monitoring for Multi-Source Conformal Prediction
Multi-source conformal prediction guarantees marginal coverage for mixtures of joint source distributions. Can a monitor that allows changing source proportions give useful warnings about coverage loss? We evaluate the credal specification test against pooled, per-source and context-aware drift tests, and coverage predictors fitted with labelled reference data. The nominal MDCP target is 90%; the measured batch-undercoverage flag is empirical coverage below 0.868, a binomial 1% rule for batches of 500. Across three encoders, 16,800 CIFAR corruption/control batches and 11,400 Camelyon17 hospital batches, the embedding hull test has pooled in-hull alarm rates of 4.4–6.7%. On Camelyon17 it detects 78–98% of flagged batches in the held-out-hospital scenarios, but many alarms have no undercoverage flag. Hospital 2 illustrates why: with ResNet-18 its mean coverage is 0.896, below the nominal target, yet only 3/200 pure-hospital batches fall below 0.868; with DINOv2 mean coverage is 0.921 and none do. The tested output-space implementation returns no result on 26–38% of hospital batches and has lower recall. Observable feature-hull membership is only a necessary implication of the joint-mixture assumption, and cannot certify coverage. Mixture-aware monitoring reduces unwanted reweighting alarms, but its usefulness as a warning about batch undercoverage depends on the shift, representation and implementation.
Authors
- S. M. Abdulla Al Mamun
Institutions
- University of Manchester (GB)
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-11-19
- DOI
- https://doi.org/10.5281/zenodo.24369423
- Primary Topic
- Adversarial Robustness in Machine Learning
- Type
- preprint