Reading the substrate, not the score: a self-calibrating friction signature that warns of fine-tuning over-fitting and maps when interventions help
Fine-tuning a language model on a narrow task can erode general capability while the held-out score still looks fine: the model keeps answering the easy validation items correctly as its reasoning collapses. This paper shows that a friction signature read directly from the model's own token log-probabilities (per-token entropy and competing routes) is an accuracy-complementary instrument that moves where the score is masked, and that the over-fitting collapse is a critical transition with a standard early-warning: per-token entropy variance and lag-1 autocorrelation rise several optimizer steps before held-out accuracy drops, while accuracy is still flat. The two distributional precursors replicate across three model families (Qwen2.5-1.5B, Phi-3.5-mini, SmolLM2-1.7B), each with multi-seed error bars. The signature is packaged as a self-calibrating monitor whose trigger is relative to each run's own opening baseline, with no global threshold and no separate calibration pass, and is validated across model families: it flags the collapse in three of three families that collapse, while a difficulty-masking gate correctly skips a fourth. The crucial contrast is that, because the failure is capability-erosion rather than over-fit to the training task, the in-task validation loss falls straight through the collapse and never warns, while the friction signature rises through it. Read across model capacity instead of training time, the same signature yields a regime-map that predicts when an instruction frame helps: it bites on pressed substrates and does nothing on frozen ones. Series position. Paper 2E in the Friction Theory paper-series, the inference regime-map and training-time early-warning node of a related cluster of companion papers. Data, notebooks, and analysis scripts are released with the paper.
Authors
- Tomas Pødenphant Lund (ORCID: https://orcid.org/0009-0000-4724-2427)
Institutions
- Aarhus University (DK)
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-06-17
- DOI
- https://doi.org/10.5281/zenodo.20562091
- Primary Topic
- Motor Control and Adaptation
- Type
- preprint