Certify, don't vote: machine-checkable certificates as the arbiter of multi-model critique

Multi-model review is now a common way to check work done with language models: several model families critique a result, and their agreement is read as confidence. We describe a method that takes agreement out of the decision. Model families generate hypotheses and objections. A claim is promoted when a mechanical arbiter accepts it (a DRAT proof checker, a Lean kernel, or re-simulation of an explicit circuit), or, through one declared fallback, when a hand-proved ordering lemma has been checked by three model families and the claim says so. Two further exceptions are declared where they occur: a lifting theorem that is machine-checked in Lean except for one premise stated in prose, corroborated by DRAT at small n, and Lean files whose proofs also trust the compiler. The method has three parts. A certification ladder states which certificate each class of claim needs and what a fallback must declare. A dated, additive claim ledger is guarded by a lint that requires a certificate reference or one of two declared alternatives. Since 2026-09-12 reviewers asked to judge a claim are asked for rankings and discriminating tests, and a verdict label they still write is recorded as data, not as a decision; since 2026-10-11 the written protocol says so (Section 5.3). Reviews of a whole manuscript before deposit still return findings, and the verdict on each finding comes from a verifier that is itself a model (Section 5.5). We report the method as one research program in exact circuit synthesis used it, and that program is the worked case. The case shows why a vote cannot be the arbiter. One model family proposed a gate-ordering constraint and a second confirmed it as sound. A third found a concrete circuit that the constraint wrongly excluded. The case also tested the method against a mathematical result rather than a benchmark. The program refuted two theorems of a published preprint: the unit-gap bound, and the classification of unit-gap circuits that the worked case is about. The author withdrew the preprint, and the withdrawal comment cites the program's counterexample to the first theorem and its certificates. Only level 1 of the ladder, plain DRAT without the added gate-ordering constraint, is implemented. Level 2, VeriPB dominance certificates, is a design: its checker is built, and there is no instance encoding, no working proof-logging solver and no dominance derivation. Our own planning notes described the nightly durability check as covering the certificate corpus. It covered one certificate. Durability today is a nightly re-verification of 32 certificates: the one the job always covered and the 31 shipped with the public bundle, added to the job on 2026-10-08. The job's log for the 31 shows all of them passing on each of the 44 nights from 2026-10-09 to 2026-11-21. Checked once on one laptop, the 31 passed in 21 s of checking (46 s for the whole run), and the checker rejected three deliberately broken proofs. The panel's process records give counts of calls and exit codes. They do not give a true-positive rate, and we do not report one. Evidence. The evidence files cited by the paper are in two public records. doi:10.5281/zenodo.24261805 holds the re-verification run and the other evidence files from the author's companion repository. doi:10.5281/zenodo.24421990 holds the cited files of the research repository, in English, and the source of this paper with every evidence marker.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-11-21
DOI
https://doi.org/10.5281/zenodo.24422003
Primary Topic
Formal Methods in Verification
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Certify, don't vote: machine-checkable certificates as the arbiter of multi-model critique

Luiz Antonio Busnello
Zenodo (CERN European Organization for Nuclear Research)
Formal Methods in Verification
preprint

Certify, don't vote: machine-checkable certificates as the arbiter of multi-model critique

Luiz Antonio Busnello
preprint en

Abstract

Multi-model review is now a common way to check work done with language models: several model families critique a result, and their agreement is read as confidence. We describe a method that takes agreement out of the decision. Model families generate hypotheses and objections. A claim is promoted when a mechanical arbiter accepts it (a DRAT proof checker, a Lean kernel, or re-simulation of an explicit circuit), or, through one declared fallback, when a hand-proved ordering lemma has been checked by three model families and the claim says so. Two further exceptions are declared where they occur: a lifting theorem that is machine-checked in Lean except for one premise stated in prose, corroborated by DRAT at small n, and Lean files whose proofs also trust the compiler. The method has three parts. A certification ladder states which certificate each class of claim needs and what a fallback must declare. A dated, additive claim ledger is guarded by a lint that requires a certificate reference or one of two declared alternatives. Since 2026-09-12 reviewers asked to judge a claim are asked for rankings and discriminating tests, and a verdict label they still write is recorded as data, not as a decision; since 2026-10-11 the written protocol says so (Section 5.3). Reviews of a whole manuscript before deposit still return findings, and the verdict on each finding comes from a verifier that is itself a model (Section 5.5). We report the method as one research program in exact circuit synthesis used it, and that program is the worked case. The case shows why a vote cannot be the arbiter. One model family proposed a gate-ordering constraint and a second confirmed it as sound. A third found a concrete circuit that the constraint wrongly excluded. The case also tested the method against a mathematical result rather than a benchmark. The program refuted two theorems of a published preprint: the unit-gap bound, and the classification of unit-gap circuits that the worked case is about. The author withdrew the preprint, and the withdrawal comment cites the program's counterexample to the first theorem and its certificates. Only level 1 of the ladder, plain DRAT without the added gate-ordering constraint, is implemented. Level 2, VeriPB dominance certificates, is a design: its checker is built, and there is no instance encoding, no working proof-logging solver and no dominance derivation. Our own planning notes described the nightly durability check as covering the certificate corpus. It covered one certificate. Durability today is a nightly re-verification of 32 certificates: the one the job always covered and the 31 shipped with the public bundle, added to the job on 2026-10-08. The job's log for the 31 shows all of them passing on each of the 44 nights from 2026-10-09 to 2026-11-21. Checked once on one laptop, the 31 passed in 21 s of checking (46 s for the whole run), and the checker rejected three deliberately broken proofs. The panel's process records give counts of calls and exit codes. They do not give a true-positive rate, and we do not report one. Evidence. The evidence files cited by the paper are in two public records. doi:10.5281/zenodo.24261805 holds the re-verification run and the other evidence files from the author's companion repository. doi:10.5281/zenodo.24421990 holds the cited files of the research repository, in English, and the source of this paper with every evidence marker.

Zenodo (CERN European Organization for Nuclear Research)
Formal Methods in Verification
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon