Reading the substrate, not the score: a self-calibrating friction signature that warns of fine-tuning over-fitting and maps when interventions help

Fine-tuning a language model on a narrow task can erode general capability while the held-out score still looks fine: the model keeps answering the easy validation items correctly as its reasoning collapses. This paper shows that a friction signature read directly from the model's own token log-probabilities (per-token entropy and competing routes) is an accuracy-complementary instrument that moves where the score is masked, and that the over-fitting collapse is a critical transition with a standard early-warning: per-token entropy variance and lag-1 autocorrelation rise several optimizer steps before held-out accuracy drops, while accuracy is still flat. The two distributional precursors replicate across three model families (Qwen2.5-1.5B, Phi-3.5-mini, SmolLM2-1.7B), each with multi-seed error bars. The signature is packaged as a self-calibrating monitor whose trigger is relative to each run's own opening baseline, with no global threshold and no separate calibration pass, and is validated across model families: it flags the collapse in three of three families that collapse, while a difficulty-masking gate correctly skips a fourth. The crucial contrast is that, because the failure is capability-erosion rather than over-fit to the training task, the in-task validation loss falls straight through the collapse and never warns, while the friction signature rises through it. Read across model capacity instead of training time, the same signature yields a regime-map that predicts when an instruction frame helps: it bites on pressed substrates and does nothing on frozen ones. Series position. Paper 2E in the Friction Theory paper-series, the inference regime-map and training-time early-warning node of a related cluster of companion papers. Data, notebooks, and analysis scripts are released with the paper.

Authors

Institutions

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-06-17
DOI
https://doi.org/10.5281/zenodo.20562091
Primary Topic
Motor Control and Adaptation
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
preprint

Reading the substrate, not the score: a self-calibrating friction signature that warns of fine-tuning over-fitting and maps when interventions help

Tomas Pødenphant Lund
Zenodo (CERN European Organization for Nuclear Research)
Motor Control and Adaptation
preprint

Reading the substrate, not the score: a self-calibrating friction signature that warns of fine-tuning over-fitting and maps when interventions help

Tomas Pødenphant Lund
preprint en

Abstract

Fine-tuning a language model on a narrow task can erode general capability while the held-out score still looks fine: the model keeps answering the easy validation items correctly as its reasoning collapses. This paper shows that a friction signature read directly from the model's own token log-probabilities (per-token entropy and competing routes) is an accuracy-complementary instrument that moves where the score is masked, and that the over-fitting collapse is a critical transition with a standard early-warning: per-token entropy variance and lag-1 autocorrelation rise several optimizer steps before held-out accuracy drops, while accuracy is still flat. The two distributional precursors replicate across three model families (Qwen2.5-1.5B, Phi-3.5-mini, SmolLM2-1.7B), each with multi-seed error bars. The signature is packaged as a self-calibrating monitor whose trigger is relative to each run's own opening baseline, with no global threshold and no separate calibration pass, and is validated across model families: it flags the collapse in three of three families that collapse, while a difficulty-masking gate correctly skips a fourth. The crucial contrast is that, because the failure is capability-erosion rather than over-fit to the training task, the in-task validation loss falls straight through the collapse and never warns, while the friction signature rises through it. Read across model capacity instead of training time, the same signature yields a regime-map that predicts when an instruction frame helps: it bites on pressed substrates and does nothing on frozen ones. Series position. Paper 2E in the Friction Theory paper-series, the inference regime-map and training-time early-warning node of a related cluster of companion papers. Data, notebooks, and analysis scripts are released with the paper.

Zenodo (CERN European Organization for Nuclear Research)
Aarhus University (DK)
Quality Education
Motor Control and Adaptation
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.