A Certificate of Release to Service for AI Agents: Pilot 0, Testing AI Models as Advisory Reviewers of Finished Work Packs
A Certificate of Release to Service is signed in aircraft maintenance only after checking that a work pack's records support the claim that work is finished. This pilot asks whether an AI model, acting as an advisory reviewer that never signs, can notice when a finished pack's evidence does not support its "done" statements. Eight tasks (five invented aviation style tasks, three derived from one software project) each had a faulty pack with one seeded fault and a clean twin. Three models reviewed each pack twice, scored by code. Every model caught, located and exactly quoted 16 of 16 seeded faults (Wilson 95% lower bound 0.81), with no invented quotes. False alarms on clean twins were 6 to 8 of 16 in run 1 and 3 to 6 of 16 after a post hoc repair of four twins. The pilot is small, self registered and not validated for real maintenance.
Authors
- Isaac Adekunle
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-11-11
- DOI
- https://doi.org/10.5281/zenodo.24125997
- Primary Topic
- Software Engineering Research
- Type
- preprint