Pre-Registered, Budget-Certified Verdicts for the Test and Evaluation of AI Systems: Certified Union Constants, Attempt Budgets, Commit-Reveal Labels and Certified Operating Envelopes
A test-and-evaluation verdict on an AI system is usually a point estimate compared with a threshold, and it can be manufactured: by re-running the evaluation until it passes, by re-reading a test after every batch, by re-phrasing the question after seeing the data, or by letting the system see its labels. We describe a protocol in which a verdict is a certificate that cannot be manufactured in these ways. Each claim is decided with a concentration margin whose logarithmic constant must cover a declared budget of looks and attempts; the evaluator certifies that constant in exact integer arithmetic before any data and refuses an under-declared policy. A canonical digest of what is asked is registered before the data; labels are committed by the source before the system commits its outputs; verdicts are three-valued (Admitted, Refuted, Open); and the same final looks certify the operating envelope, the limiting condition and the break point at no extra statistical cost. We prove a per-claim guarantee across all budgeted attempts, give a sign-pattern calibration bound 2.87 times tighter than the binned bound in a representative configuration, and report 72 verdicts in eight evaluation modes with none contrary to the declared truth. All quantities are bit-exact across architectures. Preprint, version 1, not peer reviewed. AI assistance is disclosed in the paper.
Authors
- Anthony Moreno
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-11-20
- DOI
- https://doi.org/10.5281/zenodo.24384615
- Primary Topic
- Ethics and Social Impacts of AI
- Type
- preprint