Which decisions will the next model change? A label-free budget and watchlist for AI model updates, tested on a live version change
A system that reads its decisions off a model's probabilities—a classifier, a language-model judge, a typed-decision model—inherits a problem its vendor does not solve: the model behind the identifier changes, and nobody can say in advance which past decisions the next version will reverse. When the decision is compiled from a rule over thresholded answers, the answer is in its margins, computed label-free from the snapshot: the margin of a decision is the smallest move of the answers that flips a contrastive explanation of it. It gives a budget—an update whose answers drift by at most $\delta$ changes at most $\#\{\text{margin} \le \delta\}$ decisions, published before the update exists—and a watchlist of the decisions to re-examine first. We tested both, preregistered, on a hosted model's live version change, measured at the release and again at the alias switch nine days later: on 300 documents the compiled rule changed 0 decisions both times where a single reading question changed 25 and 29 and a whole-decision question 24 and 25; the margin ranked the changes with AUROC 0.86–0.98, a 20% watchlist held 67–100% of them, and a certificate restricted to the literals the rule reads budgets 6 of 200 decisions against 71 and 138. After the update, a replayed sentinel's change rate bounds the number of changes more tightly; the margin's value is before, and in saying which.
Authors
- Oussama Chahed (ORCID: https://orcid.org/0009-0009-8062-4377)
Institutions
- Université Paris Sciences et Lettres (FR)
- Université Paris Dauphine-PSL (FR)
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-10-28
- DOI
- https://doi.org/10.5281/zenodo.23745358
- Primary Topic
- Explainable Artificial Intelligence (XAI)
- Type
- preprint