Empirical Results from 31 Rounds of Multi-Model Structural Honesty Audit

Empirical results from 31 audit rounds on the XZ Protocol, a construct-validation preregistration for structural-decision replay of the XZ Utils supply-chain compromise (CVE-2024-3094). The audit program produced 199 seat reports from five core vendor families (Anthropic, OpenAI, Google, Moonshot, Zhipu), yielding 832 structured findings (136 Class A byte-false claims). Primary result: 25 consecutive rounds (R7--R31) with Class A present, with 102 extracted Class A findings across 19 rounds and description-level or report-metadata evidence for 5 additional rounds (1 round ASSERTED from the companion Heartbeat paper). Secondary result: defect migration trajectory with recurrence across code/CI (41 findings), cross-layer consistency (23), methodology/dispatch (11), provenance/amendment (6), and unclassified (21) locus layers. The deposit includes the paper, per-round recurrence witness table, per-finding migration classification, the classification script, and the structured corpus data. Four audit rounds across three non-producer model families; carrying seat returned CONDITIONAL PASS at R4. This paper makes no effectiveness claim about any component of the Structural Honesty program.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-08-20
DOI
https://doi.org/10.5281/zenodo.22021805
Primary Topic
Intellectual Property and Patents
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Empirical Results from 31 Rounds of Multi-Model Structural Honesty Audit

Bilal Syed Arfeen
Zenodo (CERN European Organization for Nuclear Research)
Intellectual Property and Patents
article

Empirical Results from 31 Rounds of Multi-Model Structural Honesty Audit

Bilal Syed Arfeen
article en

Abstract

Empirical results from 31 audit rounds on the XZ Protocol, a construct-validation preregistration for structural-decision replay of the XZ Utils supply-chain compromise (CVE-2024-3094). The audit program produced 199 seat reports from five core vendor families (Anthropic, OpenAI, Google, Moonshot, Zhipu), yielding 832 structured findings (136 Class A byte-false claims). Primary result: 25 consecutive rounds (R7--R31) with Class A present, with 102 extracted Class A findings across 19 rounds and description-level or report-metadata evidence for 5 additional rounds (1 round ASSERTED from the companion Heartbeat paper). Secondary result: defect migration trajectory with recurrence across code/CI (41 findings), cross-layer consistency (23), methodology/dispatch (11), provenance/amendment (6), and unclassified (21) locus layers. The deposit includes the paper, per-round recurrence witness table, per-finding migration classification, the classification script, and the structured corpus data. Four audit rounds across three non-producer model families; carrying seat returned CONDITIONAL PASS at R4. This paper makes no effectiveness claim about any component of the Structural Honesty program.

Zenodo (CERN European Organization for Nuclear Research)
Reduced inequalities
Openalex Percentile: Top 5%
Intellectual Property and Patents
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Empirical Results from 31 Rounds of Multi-Model Structural Honesty Audit — Bilal Syed Arfeen · Zenodo (CERN European Organization for Nuclear Research) (2026) | TGRS Research Map | TGRS