Surprise Is Not Importance: Criticality Read from Hidden States Gates Sleep-Like Consolidation

A model that learns new facts every night forgets what it does not rehearse, so a fixed replay budget is a choice of what matters. Surprise is the usual guide; we show it is the wrong one. A small probe on the frozen residual stream of Qwen2.5-0.5B-Instruct scores how critical a fact is (health, safety, money, legal risk) at AUROC 0.99, while answer-loss surprise scores 0.53 and falls for lexical traps. Used to choose what to replay over ten nights of LoRA consolidation, the probe retains 20.3 of 30 critical facts, against 12.7 for random replay and 10.3 for surprise, on par with a label oracle. The signal is semantic, not lexical: on an independent bilingual exam outside the training templates the probe keeps 0.98–0.99 while word TF-IDF drops to 0.65. Two limits are reported: off-the-shelf sentence encoders match the probe, and criticality that depends on facts stored in the adapter is read only weakly (0.57). All data are synthetic or LLM-written; three-seed contrasts are descriptive.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-11-11
DOI
https://doi.org/10.5281/zenodo.24131187
Primary Topic
Topic Modeling
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Surprise Is Not Importance: Criticality Read from Hidden States Gates Sleep-Like Consolidation

Sebastian Goin del Rio
Zenodo (CERN European Organization for Nuclear Research)
Topic Modeling
preprint

Surprise Is Not Importance: Criticality Read from Hidden States Gates Sleep-Like Consolidation

Sebastian Goin del Rio
preprint en

Abstract

A model that learns new facts every night forgets what it does not rehearse, so a fixed replay budget is a choice of what matters. Surprise is the usual guide; we show it is the wrong one. A small probe on the frozen residual stream of Qwen2.5-0.5B-Instruct scores how critical a fact is (health, safety, money, legal risk) at AUROC 0.99, while answer-loss surprise scores 0.53 and falls for lexical traps. Used to choose what to replay over ten nights of LoRA consolidation, the probe retains 20.3 of 30 critical facts, against 12.7 for random replay and 10.3 for surprise, on par with a label oracle. The signal is semantic, not lexical: on an independent bilingual exam outside the training templates the probe keeps 0.98–0.99 while word TF-IDF drops to 0.65. Two limits are reported: off-the-shelf sentence encoders match the probe, and criticality that depends on facts stored in the adapter is read only weakly (0.57). All data are synthetic or LLM-written; three-seed contrasts are descriptive.

Zenodo (CERN European Organization for Nuclear Research)
Topic Modeling
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon