What Breaks in Small Generative Process Verifiers: Verdict Aggregation, Calibration Limits, and Discriminative–Generative Cascades

We study a 1.5B generative process verifier (DeepSeek-R1-Distill-Qwen-1.5B with a fixed step-judgment format) on PRM800K and find that its failures lie mostly where accuracy numbers do not point. (1) Step accuracy is a base-rate mirage. A trivial all-"correct" judge scores 0.904 on the test steps; our SFT verifier scores 0.897 while flagging 15 of 1,857 steps, one correctly. Its verdicts mirror the majority class, and neither GRPO (every sampled group agrees, so advantages vanish) nor decode-time derivation repairs this. Training on multipath-consistent traces does (solution accuracy 0.417 → 0.583, McNemar p ≈ 10^-5). (2) The verifier knows more than it says. Verdict-token probabilities reach step AUROC 0.81, and noisy-OR aggregation over them gives 0.883 solution accuracy at ECE 0.092 with no fitting (SFT v2, 103 test traces), while the emitted confidence is two-valued by target design. (3) Step-label interference is real but small. Teacher-forcing an "incorrect" judgment at step k leaves later hard verdicts unchanged (CIs exclude drag increases above 0.4 points) but raises later steps' P(incorrect) by about one probability point (SFT v2: +0.9, 95% CI [+0.1, +1.9]). (4) Cascades trade cost for accuracy. Escalating only the traces a discriminative PRM is unsure about to the token-probability readout reaches 0.820 accuracy on 891 traces (the 103 test traces plus 788 assembled from their candidate pools) while calling the generative verifier on 20.5% of them. This beats the discriminative PRM alone (0.744; +7.7 points, problem-clustered 95% CI [+3.6, +12.3]) and matches running the readout on every trace within noise (0.773; +4.6 points, CI [-1.7, +11.4]) at a fifth of its generative cost.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-10-28
DOI
https://doi.org/10.5281/zenodo.23719740
Primary Topic
Adversarial Robustness in Machine Learning
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
preprint

What Breaks in Small Generative Process Verifiers: Verdict Aggregation, Calibration Limits, and Discriminative–Generative Cascades

Qiwei Li
Zenodo (CERN European Organization for Nuclear Research)
Adversarial Robustness in Machine Learning
preprint

What Breaks in Small Generative Process Verifiers: Verdict Aggregation, Calibration Limits, and Discriminative–Generative Cascades

Qiwei Li
preprint en

Abstract

We study a 1.5B generative process verifier (DeepSeek-R1-Distill-Qwen-1.5B with a fixed step-judgment format) on PRM800K and find that its failures lie mostly where accuracy numbers do not point. (1) Step accuracy is a base-rate mirage. A trivial all-"correct" judge scores 0.904 on the test steps; our SFT verifier scores 0.897 while flagging 15 of 1,857 steps, one correctly. Its verdicts mirror the majority class, and neither GRPO (every sampled group agrees, so advantages vanish) nor decode-time derivation repairs this. Training on multipath-consistent traces does (solution accuracy 0.417 → 0.583, McNemar p ≈ 10^-5). (2) The verifier knows more than it says. Verdict-token probabilities reach step AUROC 0.81, and noisy-OR aggregation over them gives 0.883 solution accuracy at ECE 0.092 with no fitting (SFT v2, 103 test traces), while the emitted confidence is two-valued by target design. (3) Step-label interference is real but small. Teacher-forcing an "incorrect" judgment at step k leaves later hard verdicts unchanged (CIs exclude drag increases above 0.4 points) but raises later steps' P(incorrect) by about one probability point (SFT v2: +0.9, 95% CI [+0.1, +1.9]). (4) Cascades trade cost for accuracy. Escalating only the traces a discriminative PRM is unsure about to the token-probability readout reaches 0.820 accuracy on 891 traces (the 103 test traces plus 788 assembled from their candidate pools) while calling the generative verifier on 20.5% of them. This beats the discriminative PRM alone (0.744; +7.7 points, problem-clustered 95% CI [+3.6, +12.3]) and matches running the readout on every trace within noise (0.773; +4.6 points, CI [-1.7, +11.4]) at a fifth of its generative cost.

Zenodo (CERN European Organization for Nuclear Research)
Adversarial Robustness in Machine Learning
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon