Empirical Results from 31 Rounds of Multi-Model Structural Honesty Audit
Empirical results from 31 audit rounds on the XZ Protocol, a construct-validation preregistration for structural-decision replay of the XZ Utils supply-chain compromise (CVE-2024-3094). The audit program produced 199 seat reports from five core vendor families (Anthropic, OpenAI, Google, Moonshot, Zhipu), yielding 832 structured findings (136 Class A byte-false claims). Primary result: 25 consecutive rounds (R7--R31) with Class A present, with 102 extracted Class A findings across 19 rounds and description-level or report-metadata evidence for 5 additional rounds (1 round ASSERTED from the companion Heartbeat paper). Secondary result: defect migration trajectory with recurrence across code/CI (41 findings), cross-layer consistency (23), methodology/dispatch (11), provenance/amendment (6), and unclassified (21) locus layers. The deposit includes the paper, per-round recurrence witness table, per-finding migration classification, the classification script, and the structured corpus data. Four audit rounds across three non-producer model families; carrying seat returned CONDITIONAL PASS at R4. This paper makes no effectiveness claim about any component of the Structural Honesty program.
Authors
- Bilal Syed Arfeen
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-08-20
- DOI
- https://doi.org/10.5281/zenodo.22021805
- Primary Topic
- Intellectual Property and Patents
- Type
- article
- Field-Weighted Citation Impact
- 0.00