Which decisions will the next model change? A label-free budget and watchlist for AI model updates, tested on a live version change

A system that reads its decisions off a model's probabilities—a classifier, a language-model judge, a typed-decision model—inherits a problem its vendor does not solve: the model behind the identifier changes, and nobody can say in advance which past decisions the next version will reverse. When the decision is compiled from a rule over thresholded answers, the answer is in its margins, computed label-free from the snapshot: the margin of a decision is the smallest move of the answers that flips a contrastive explanation of it. It gives a budget—an update whose answers drift by at most $\delta$ changes at most $\#\{\text{margin} \le \delta\}$ decisions, published before the update exists—and a watchlist of the decisions to re-examine first. We tested both, preregistered, on a hosted model's live version change, measured at the release and again at the alias switch nine days later: on 300 documents the compiled rule changed 0 decisions both times where a single reading question changed 25 and 29 and a whole-decision question 24 and 25; the margin ranked the changes with AUROC 0.86–0.98, a 20% watchlist held 67–100% of them, and a certificate restricted to the literals the rule reads budgets 6 of 200 decisions against 71 and 138. After the update, a replayed sentinel's change rate bounds the number of changes more tightly; the margin's value is before, and in saying which.

Authors

Institutions

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-10-28
DOI
https://doi.org/10.5281/zenodo.23745358
Primary Topic
Explainable Artificial Intelligence (XAI)
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
preprint

Which decisions will the next model change? A label-free budget and watchlist for AI model updates, tested on a live version change

Oussama Chahed
2 links
Zenodo (CERN European Organization for Nuclear Research)
Explainable Artificial Intelligence (XAI)
preprint

Which decisions will the next model change? A label-free budget and watchlist for AI model updates, tested on a live version change

Oussama Chahed
preprint en

Abstract

A system that reads its decisions off a model's probabilities—a classifier, a language-model judge, a typed-decision model—inherits a problem its vendor does not solve: the model behind the identifier changes, and nobody can say in advance which past decisions the next version will reverse. When the decision is compiled from a rule over thresholded answers, the answer is in its margins, computed label-free from the snapshot: the margin of a decision is the smallest move of the answers that flips a contrastive explanation of it. It gives a budget—an update whose answers drift by at most $\delta$ changes at most $\#\{\text{margin} \le \delta\}$ decisions, published before the update exists—and a watchlist of the decisions to re-examine first. We tested both, preregistered, on a hosted model's live version change, measured at the release and again at the alias switch nine days later: on 300 documents the compiled rule changed 0 decisions both times where a single reading question changed 25 and 29 and a whole-decision question 24 and 25; the margin ranked the changes with AUROC 0.86–0.98, a 20% watchlist held 67–100% of them, and a certificate restricted to the literals the rule reads budgets 6 of 200 decisions against 71 and 138. After the update, a replayed sentinel's change rate bounds the number of changes more tightly; the margin's value is before, and in saying which.

Zenodo (CERN European Organization for Nuclear Research)
Université Paris Sciences et Lettres (FR), Université Paris Dauphine-PSL (FR)
Explainable Artificial Intelligence (XAI)
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon