Verification Debt in AI-Generated Mathematics: An Empirical Archival and Adversarial Pilot

The prospect of AI producing research claims faster than researchers can validate themrequires a measurement programme rather than extrapolation from manuscript counts. Weundertake a reproducible pilot centred on the October 2026 openai/math mathematicalmanuscript archive and two independent verification activities. We extract a public revisionchronology, distinguish dependent withdrawals from independent failures, independentlyrecheck a source-level citation misidentification and an extraction-induced false alarm, andrun two implementations of a documented signed-count correction. A repository updatecontaining three linked withdrawals, fourteen revised manuscripts, and thirteen furthercitation/version changes was committed 31 hours 5 minutes after the initial public commit;this measures archive latency, not the labour required to discover the defects. The threewithdrawals share one root sign error and thus cannot be treated as independent Bernoullifailures. In a purpose-built control suite, seventeen targeted corruptions were detected andfive out-of-scope semantic strengthenings were not; these engineered counts are a scope test,not an empirical estimate of natural error-detection sensitivity. We provide dated sourceidentifiers, a machine-readable event ledger, a source manifest with Git blob identifiers, and anoffline reproducer. Neither independent reviewer labour times nor downstream scientific-usetimes can currently be estimated from the public material inspected. We give a preregistrableprospective protocol capable of measuring both. The pilot demonstrates the feasibility ofverification observability while refusing population-level error-rate claims unsupported by thedata.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-11-15
DOI
https://doi.org/10.5281/zenodo.24247093
Primary Topic
Academic Publishing and Open Access
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Verification Debt in AI-Generated Mathematics: An Empirical Archival and Adversarial Pilot

Tony Newton
Zenodo (CERN European Organization for Nuclear Research)
Academic Publishing and Open Access
preprint

Verification Debt in AI-Generated Mathematics: An Empirical Archival and Adversarial Pilot

Tony Newton
preprint en

Abstract

The prospect of AI producing research claims faster than researchers can validate themrequires a measurement programme rather than extrapolation from manuscript counts. Weundertake a reproducible pilot centred on the October 2026 openai/math mathematicalmanuscript archive and two independent verification activities. We extract a public revisionchronology, distinguish dependent withdrawals from independent failures, independentlyrecheck a source-level citation misidentification and an extraction-induced false alarm, andrun two implementations of a documented signed-count correction. A repository updatecontaining three linked withdrawals, fourteen revised manuscripts, and thirteen furthercitation/version changes was committed 31 hours 5 minutes after the initial public commit;this measures archive latency, not the labour required to discover the defects. The threewithdrawals share one root sign error and thus cannot be treated as independent Bernoullifailures. In a purpose-built control suite, seventeen targeted corruptions were detected andfive out-of-scope semantic strengthenings were not; these engineered counts are a scope test,not an empirical estimate of natural error-detection sensitivity. We provide dated sourceidentifiers, a machine-readable event ledger, a source manifest with Git blob identifiers, and anoffline reproducer. Neither independent reviewer labour times nor downstream scientific-usetimes can currently be estimated from the public material inspected. We give a preregistrableprospective protocol capable of measuring both. The pilot demonstrates the feasibility ofverification observability while refusing population-level error-rate claims unsupported by thedata.

Zenodo (CERN European Organization for Nuclear Research)
Academic Publishing and Open Access
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon