Finite-sample decision risk in unseeded quantum compiler benchmarking

Version 4 — audited release. Archival technical preprint; not submitted to a venue. Quantum compiler benchmarks are used to accept or reject changes to production transpilers, but the compilers they measure are stochastic. At the pinned revision of IBM's Benchpress benchmark suite, its Qiskit gym compiles without passing seed_transpiler, so a reported gate count is one draw from an unmeasured distribution — while the BQSKit gym in the same repository does seed its compiler. This study quantifies the consequence as finite-sample decision risk: the probability that a regression verdict computed from k runs per version disagrees with the verdict implied by the long-run mean of the same measurement. Design. 39 QASMBench circuits (28–420 qubits) were transpiled under Qiskit 1.4.3 → 2.0.0 with 200 recorded seeds per arm — 15,600 primary transpilations — under a pre-registered analysis with a +10% threshold and a three-run decision rule. Of the 39 circuits, 36 resolved, and 26 were eligible after pre-registered boundary exclusions. Primary endpoint. Over the 26 eligible circuits, 12/26 have a decision-risk interval that excludes zero. 7/26 carry a risk of at least 5% and 4/26 at least 10%. Over the same eligible set the risk distribution has median 0, p75 6.9%, p90 14.2% and maximum 17.3%. These are descriptive counts of these 26 circuits; the paper does not advance a suite-level rate, and explains why the effective sample size does not support one. Ambiguity band. For the version pair and residual model stated in the paper, the median width of the interval of true change over which the three-run rule's call probability runs from 5% to 95% is approximately 10.9 percentage points. The band is constructed from the measured per-seed residuals of this specific version pair under a stated multiplicative model; it is not claimed to be independent of the version pair, and the modelling conventions are stated rather than assumed. What is included. The reproducibility archive contains the exact frozen source tree at the audited commit: the raw per-seed measurements, the analysis code, the manuscript, the pre-registration, the correction ledgers, and the assurance apparatus itself. The endpoint is reconstructed from the raw observations on every verification run rather than read from a summary table. Assurance and its boundaries. The repository carries a thirteen-stage read-only verifier and a 44-fixture mutation suite in which every corruption must be rejected by a named stage under the invariant that stage declares. An independent hostile audit of this release found no release-blocking defect. The boundaries of that assurance are documented rather than implied: the published-PDF check binds word content in both directions but does not establish semantic word order, the registered manuscript surface is a declared Markdown model rather than the rendered page, and a documented human reading of the built PDF remains a release requirement — performed for this version. No claim of exhaustive or formal verification is made. Corrections are kept, not removed. Earlier versions of this work stated claims that were subsequently withdrawn or corrected, including a suite-level rate, a reported correlation between stochastic compilation and risk, and a description of the ambiguity band as version-pair-free. Those corrections remain documented in the paper and in the correction ledgers shipped with the archive. Scope limits. The study measures one version pair, one topology family and one benchmark suite revision. It does not identify a causal compiler mechanism, does not establish that decision risk is monotone in any quantity, does not claim that setting seed_transpiler makes compilation deterministic in general, and does not reproduce the historical protocol of Qiskit issue #14402. Integrity Source commit: e663912278baeda6d24787b37ad8a04243a4280b (tag v4.0.0) paper.pdf SHA-256: 19e37c1954368928475b4ef51d5f8ca8a28d41b896932de6df8e4edf0fe9c7a5 qvalidation-e663912278ba.zip SHA-256: 07126ddda949ffb8a4d44b5763df5e22ff8488b20aac958bad088653b4e19941 — the exact 1,265-file frozen Git tree Raw pre-registered evidence tree: 3114a270ccd78b0daaa2a1bc451dd15155d55c13