Surprise Is Not Importance: Criticality Read from Hidden States Gates Sleep-Like Consolidation
A model that learns new facts every night forgets what it does not rehearse, so a fixed replay budget is a choice of what matters. Surprise is the usual guide; we show it is the wrong one. A small probe on the frozen residual stream of Qwen2.5-0.5B-Instruct scores how critical a fact is (health, safety, money, legal risk) at AUROC 0.99, while answer-loss surprise scores 0.53 and falls for lexical traps. Used to choose what to replay over ten nights of LoRA consolidation, the probe retains 20.3 of 30 critical facts, against 12.7 for random replay and 10.3 for surprise, on par with a label oracle. The signal is semantic, not lexical: on an independent bilingual exam outside the training templates the probe keeps 0.98–0.99 while word TF-IDF drops to 0.65. Two limits are reported: off-the-shelf sentence encoders match the probe, and criticality that depends on facts stored in the adapter is read only weakly (0.57). All data are synthetic or LLM-written; three-seed contrasts are descriptive.
Authors
- Sebastian Goin del Rio (ORCID: https://orcid.org/0009-0006-1147-9995)
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-11-11
- DOI
- https://doi.org/10.5281/zenodo.24131187
- Primary Topic
- Topic Modeling
- Type
- preprint