Detection Is Not Action: The Detectability Boundary of Pre-training Data-Readiness Gating
Many high-stakes AI failures trace to the training data rather than the model, yet teams still treat pre-training data validation as a checklist and rarely wire its signals to a go/no-go decision or to a governance framework. We study the detectability boundary of that validation: what a pre-training data-readiness gate catches, what it misses, and why a flagged source is often not blocked. We reuse a validation gate from a prior single-domain system and generalize it across three high-stakes domains (healthcare, finance, and government) on the real MIMIC-IV, SEC EDGAR, and CMS Open Payments datasets, injecting controlled defects across simulated sources. A composite readiness score detects 0.76 of degraded sources (95% CI [0.74, 0.76]) but withholds only 0.42; the 0.34 gap (95% CI [0.32, 0.35]) traces to a single averaging step, and we show it is fixable but privacy-sensitive. The grid is small by design (six sources per domain, two degraded, five seeds), so the intervals describe this controlled setting, not field variability. We map the privacy-utility curve of validating on differentially private summaries, prove a blind spot for symmetric label noise on balanced targets, and add a confident-learning label detector that runs on DP-noised summaries and keeps its recall down to epsilon = 0.1. We compare against Great Expectations and Deequ, measure how poorly thresholds port across domains, and map the gate to the NIST AI RMF MEASURE function. We report the negatives, among them a near-chance inter-rater reliability on fine-grained incident coding. What is new is the integration of production incidents, a private benchmarked gate, and a governance mapping, not any one of those parts.
Authors
- Gargi Desai (ORCID: https://orcid.org/0009-0007-9491-2949)
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-11-13
- DOI
- https://doi.org/10.5281/zenodo.24204022
- Primary Topic
- Ethics and Social Impacts of AI
- Type
- preprint