MARAM: a free in-weights verifier for parallel reasoning streams: selection beats voting, matches beam at equal tokens, and pruning doesn't help

We train decoder-only language models (111,301,634 and 339,267,586 parameters, “125M” and “350M”) from scratch on a single 48 GB GPU in a block-interleaved layout where K = 3 reasoning streams share a prompt under a tree attention mask with bit-exact stream isolation (verified by an exact-equality test on every reported checkpoint), and a checkpoint token with cross-stream visibility feeds a small head (1,181,954 parameters at 125M, 2,100,226 at 350M) that predicts per-stream correctness. Multi-stream training costs no base-LM quality we can measure: greedy accuracy on held-out 2–5-step arithmetic is 0.7470 [0.727, 0.766] for the 125M MARAM model and 0.7535 [0.734, 0.772] for a flat causal model with the same architecture and token budget. The judge trained on teacher traces transfers only partly to the model’s own outputs (its probabilities on them are badly calibrated and its ranking inside native decoding is near chance); a head-only on-policy fine-tune (405 s at 125M, 1,057 s at 350M) fixes it (AUROC on held-out self-generated streams 0.8097 → 0.8937 at 125M, 0.8215 → 0.9542 at 350M). At 350M on held-out 5–7-step arithmetic, judge selection over three streams reaches 0.7755 [0.757, 0.793] against 0.7095 [0.689, 0.729] for greedy, 0.7325 [0.713, 0.751] for a matched-token majority vote, 0.7625 [0.743, 0.781] for beam-2 at a smaller token budget (366 vs 425 tokens) and 0.8070 [0.789, 0.824] for beam-3 at 1.45× the tokens. Selection, not pruning, carries the gain (selector-only 0.7730 [0.754, 0.791]; gated pruning adds 0.0025 with 0.06 kills per problem). Judge quality is the bottleneck at 125M, where the judge cannot rank same-strategy streams (AUROC 0.498 on its own diverged streams on the same set) and selection is below the vote on identical samples (0.7290 vs 0.7540 on 2–5-step arithmetic); at 350M it ranks them (0.854). The trained K and N do not transfer to inference: decoding with four streams or 128-token blocks collapses accuracy, and retuning the pruning thresholds moves it by at most 0.9 points. Total cost 149.98 GPU-hours. Model weights, tokenizer, inference code and held-out problem files are released on Hugging Face under a research-only licence (link under Related works). Training code is not released.