Capability Emergence Can Be Forecast: Per-Seed, In Advance, With Calibrated Intervals, Certified False Alarms, and a Blind Pre-Registered Gate

Emergent capabilities are widely treated as unpredictable: loss improves smoothly while abilities appear abruptly. Prior work offers early-warning indicators but never scores them as forecasts: no lead time at controlled false-alarm rate, no calibration, no negatives, no blind tests. We supply that discipline and show that, in grokking model systems and small language models, emergence timing is forecastable per run, in advance, with calibrated uncertainty. Across 30 transformers at identical configuration, the formation time of the previous-token head forecasts each seed's induction-head emergence at Spearman rho=0.977 with median lead 975 steps (~15% of training); a best-case loss rule ties the ranking with 50-step lead (a nowcast). Conformal intervals covered 15/15 held-out seeds, and the frozen rule passed blind pre-registered gates on TWO never-seen configurations (10/10 and 9/10 coverage). A trap-language rung then attacked our own rule as pre-registered: where previous-token context pays for the task itself, the bare precursor false-alarms on 10/10 capability-blocked runs, while the mechanism-composed conjunction is certified in both language classes (0 false alarms) and times emergence at rho=1.000. Finally, a gap-origin study broke the fixed offset (both lr and batch move the gap ~2.3x; no external clock owns it) and revealed the law beneath: across 80 valid-anchor runs the anchor fires at 0.843 of time-to-emergence -- t_event ~= 1.19 x t_anchor -- and this multiplicative rule passed its own blind gate (5/5) at a third unseen configuration. False alarms are certified against 33 manufactured negatives. The precursor leads across 3 public model families (Pythia, OLMo, OLMo-2; 7 suites), with OLMo-2 at 1B tokens showing precursor formed, capability absent. Four pre-registered kill criteria fired and are reported. Every freeze precedes its data in a public commit chain.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-07-17
DOI
https://doi.org/10.5281/zenodo.21419923
Primary Topic
Ecosystem dynamics and resilience
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
preprint

Capability Emergence Can Be Forecast: Per-Seed, In Advance, With Calibrated Intervals, Certified False Alarms, and a Blind Pre-Registered Gate

Gunner Levi Howe
Zenodo (CERN European Organization for Nuclear Research)
Ecosystem dynamics and resilience
preprint

Capability Emergence Can Be Forecast: Per-Seed, In Advance, With Calibrated Intervals, Certified False Alarms, and a Blind Pre-Registered Gate

Gunner Levi Howe
preprint en

Abstract

Emergent capabilities are widely treated as unpredictable: loss improves smoothly while abilities appear abruptly. Prior work offers early-warning indicators but never scores them as forecasts: no lead time at controlled false-alarm rate, no calibration, no negatives, no blind tests. We supply that discipline and show that, in grokking model systems and small language models, emergence timing is forecastable per run, in advance, with calibrated uncertainty. Across 30 transformers at identical configuration, the formation time of the previous-token head forecasts each seed's induction-head emergence at Spearman rho=0.977 with median lead 975 steps (~15% of training); a best-case loss rule ties the ranking with 50-step lead (a nowcast). Conformal intervals covered 15/15 held-out seeds, and the frozen rule passed blind pre-registered gates on TWO never-seen configurations (10/10 and 9/10 coverage). A trap-language rung then attacked our own rule as pre-registered: where previous-token context pays for the task itself, the bare precursor false-alarms on 10/10 capability-blocked runs, while the mechanism-composed conjunction is certified in both language classes (0 false alarms) and times emergence at rho=1.000. Finally, a gap-origin study broke the fixed offset (both lr and batch move the gap ~2.3x; no external clock owns it) and revealed the law beneath: across 80 valid-anchor runs the anchor fires at 0.843 of time-to-emergence -- t_event ~= 1.19 x t_anchor -- and this multiplicative rule passed its own blind gate (5/5) at a third unseen configuration. False alarms are certified against 33 manufactured negatives. The precursor leads across 3 public model families (Pythia, OLMo, OLMo-2; 7 suites), with OLMo-2 at 1B tokens showing precursor formed, capability absent. Four pre-registered kill criteria fired and are reported. Every freeze precedes its data in a public commit chain.

Zenodo (CERN European Organization for Nuclear Research)
Peace, Justice and strong institutions
Ecosystem dynamics and resilience
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Capability Emergence Can Be Forecast: Per-Seed, In Advance, With Calibrated Intervals, Certified False Alarms, and a Blind Pre-Registered Gate — Gunner Levi Howe · Zenodo (CERN European Organization for Nuclear Research) (2026) | TGRS Research Map | TGRS