Receipt-Chained Evaluation: Reproducible Benchmark Claims from Raw Model Replies

Benchmark numbers in machine-learning papers are usually reported as aggregates whose provenance stops at a table. We describe an evaluation artifact in which every reported figure is a function of a hash-chained log of raw model replies, an item bank, and a deterministic derivation that a reader can rerun offline. We make three contributions: (i) a receipt format that binds each run to the exact item bank it was scored against; (ii) a correctness definition that re-parses the raw reply label rather than trusting a harness-side flag, and the documented disagreement this produces; and (iii) a verification target that fails the build when any derived number, chain hash, or item-bank hash does not match. We report what the verifier covers, what it deliberately does not, and the failure modes we found while making the bundle cold-clone clean. Artifact. This record contains the compiled paper (paper/main.pdf), the full LaTeX source, the item banks, the hash-chained receipt log, derive.py and verify.py, and a deterministic bundle.sh that rebuilds and re-verifies the bundle from a cold clone. Run make verify to check every derived number against the receipts. Note on Table 1. The shipped receipts are a synthetic fixture that exercises the mechanism end to end; they are not an empirical benchmark result. The paper says so under the table. Paper text and figures: CC BY 4.0. Code: MIT.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-11-14
DOI
https://doi.org/10.5281/zenodo.24223927
Primary Topic
Scientific Computing and Data Management
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Receipt-Chained Evaluation: Reproducible Benchmark Claims from Raw Model Replies

Ali Shuaib
Zenodo (CERN European Organization for Nuclear Research)
Scientific Computing and Data Management
preprint

Receipt-Chained Evaluation: Reproducible Benchmark Claims from Raw Model Replies

Ali Shuaib
preprint en

Abstract

Benchmark numbers in machine-learning papers are usually reported as aggregates whose provenance stops at a table. We describe an evaluation artifact in which every reported figure is a function of a hash-chained log of raw model replies, an item bank, and a deterministic derivation that a reader can rerun offline. We make three contributions: (i) a receipt format that binds each run to the exact item bank it was scored against; (ii) a correctness definition that re-parses the raw reply label rather than trusting a harness-side flag, and the documented disagreement this produces; and (iii) a verification target that fails the build when any derived number, chain hash, or item-bank hash does not match. We report what the verifier covers, what it deliberately does not, and the failure modes we found while making the bundle cold-clone clean. Artifact. This record contains the compiled paper (paper/main.pdf), the full LaTeX source, the item banks, the hash-chained receipt log, derive.py and verify.py, and a deterministic bundle.sh that rebuilds and re-verifies the bundle from a cold clone. Run make verify to check every derived number against the receipts. Note on Table 1. The shipped receipts are a synthetic fixture that exercises the mechanism end to end; they are not an empirical benchmark result. The paper says so under the table. Paper text and figures: CC BY 4.0. Code: MIT.

Zenodo (CERN European Organization for Nuclear Research)
Scientific Computing and Data Management
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon