When the Confidence Score Doesn't Describe the Output: A Five-Month Audit of a Deployed Biomedical AI System

AI systems for biomedical research increasingly report their own confidence: grounding ratios, citation grades, evidence scores. These self-reported metrics are almost never audited against the output they describe. We report a retrospective audit of 5,504 immutable, signed analysis records produced by a deployed biomedical hypothesis-generation system over five months (June to October 2026). In 87.3% of runs, the grounding value shown to users diverged from the system’s own trust-adjusted grounding value computed on the same record (mean 0.757 against 0.347). In 475 publicly reachable analyses the interface reported complete grounding while the trust-adjusted value was zero; these runs were not empty, carrying between one and four enriched pathways. The generating defect resolved in August 2026 through unrelated changes, but the published records continued to display the incorrect value until this audit. Over the same period the system’s own share-readiness flag was set on 0.56% of runs while the interface displayed complete grounding on 46.7%. A separate reconciliation of the same system’s cryptographic commitment ledger found a second, independent failure: a maintenance job deleted prediction records after they had been committed to Bitcoin-anchored Merkle roots, leaving 18 committed leaves across 3 roots permanently unresolvable on the system’s own public verification page. We argue this failure class is structural rather than incidental, propose a minimal audit protocol applicable to any system that publishes self-reported confidence, and release the audit queries. No stored records were altered.