Review-Conditioned Reliability Cards: Making the Denominator Visible in Human–AI Review

An AI system can be 90% accurate overall while only 53% of its suggestions in a human review queue are correct. Selective routing can cause this difference without model degradation. We propose a review-conditioned reliability card that names its audited population and denominator and expires after relevant context changes. We combine an implemented local prototype, a deterministic diagnostic, a descriptive reanalysis of public data from a published 198-participant study, and a prospective evaluation protocol. The reanalysis contextualizes why communication matters; it does not test our card. The candidate contribution is a narrow interface extension of established reporting and deferral work, not new probability theory or demonstrated human-performance improvement. We conducted no new human study. AI assistance. OpenAI-based AI assistants assisted research-question development, related-work search, manuscript drafting, prototype and analysis-code generation, deterministic diagnostic design, secondary-data analysis, and artifact checks. The public human-study data were produced by Bondi et al.; no new human-participant study was conducted. Exact assistant model versions are not documented in the supplied package. Materials and attribution. The manuscript includes an attributed secondary visualization of public aggregate data from Bondi et al. (2022), Role of Human-AI Interaction in Selective Prediction, https://doi.org/10.1609/aaai.v36i5.20465. Source data: DeepMind Technologies Limited, CC BY 4.0. The visualization is a modification; no endorsement is implied. Research draft, revision 3. Not peer reviewed. Release status. The uploader approved public release of this revision on 2026-11-08. Author-review and submission-approval notes in the PDFs were written during draft preparation. This is a preprint and has not been peer reviewed.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-11-08
DOI
https://doi.org/10.5281/zenodo.24058762
Primary Topic
Explainable Artificial Intelligence (XAI)
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Review-Conditioned Reliability Cards: Making the Denominator Visible in Human–AI Review

Musa Emin Ozdem, Rumeysa Egri
2 links
Zenodo (CERN European Organization for Nuclear Research)
Explainable Artificial Intelligence (XAI)
preprint

Review-Conditioned Reliability Cards: Making the Denominator Visible in Human–AI Review

Musa Emin Ozdem, Rumeysa Egri
preprint en

Abstract

An AI system can be 90% accurate overall while only 53% of its suggestions in a human review queue are correct. Selective routing can cause this difference without model degradation. We propose a review-conditioned reliability card that names its audited population and denominator and expires after relevant context changes. We combine an implemented local prototype, a deterministic diagnostic, a descriptive reanalysis of public data from a published 198-participant study, and a prospective evaluation protocol. The reanalysis contextualizes why communication matters; it does not test our card. The candidate contribution is a narrow interface extension of established reporting and deferral work, not new probability theory or demonstrated human-performance improvement. We conducted no new human study. AI assistance. OpenAI-based AI assistants assisted research-question development, related-work search, manuscript drafting, prototype and analysis-code generation, deterministic diagnostic design, secondary-data analysis, and artifact checks. The public human-study data were produced by Bondi et al.; no new human-participant study was conducted. Exact assistant model versions are not documented in the supplied package. Materials and attribution. The manuscript includes an attributed secondary visualization of public aggregate data from Bondi et al. (2022), Role of Human-AI Interaction in Selective Prediction, https://doi.org/10.1609/aaai.v36i5.20465. Source data: DeepMind Technologies Limited, CC BY 4.0. The visualization is a modification; no endorsement is implied. Research draft, revision 3. Not peer reviewed. Release status. The uploader approved public release of this revision on 2026-11-08. Author-review and submission-approval notes in the PDFs were written during draft preparation. This is a preprint and has not been peer reviewed.

Zenodo (CERN European Organization for Nuclear Research)
Explainable Artificial Intelligence (XAI)
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon