Reading the substrate, not the score: a self-calibrating friction signature that warns of fine-tuning over-fitting and maps when interventions help

Early warning of a training-time collapse has so far been read from a model’s internals. This paper shows the same precursor is legible in the model’s own output distribution: per-token entropy variance and lag-1 autocorrelation, computed from the log-probabilities any inference interface already returns, rise before held-out accuracy drops when narrow fine-tuning begins to erode general reasoning. The two precursors replicate across three model families (Qwen2.5-1.5B, Phi-3.5-mini, SmolLM2-1.7B) with multi-seed error bars, and the lead over the accuracy drop runs from several optimizer steps to coincident depending on the model. The signature is packaged as a self-calibrating monitor whose trigger is relative to each run’s own opening baseline, so there is no global threshold and no separate calibration pass. It flags the collapse in three of three families that collapse, while a difficulty-masking gate correctly skips a fourth. The load-bearing contrast is that the failure is capability-erosion rather than over-fit to the training task: the in-task validation loss falls straight through the collapse and never warns, and a ceiling-masked accuracy probe never moves, while the friction signature rises through the transition. An estimator ablation shows the advantage of the distributional moments over a scalar entropy level is transfer rather than lead — the level matches them on two families and fails on the third. Read across model capacity instead of training time, the same signature yields a regime-map that predicts when an instruction frame helps: it bites on pressed substrates and does nothing on frozen ones. A published account of entropy collapse proves it a first-order transition for feedback-amplified systems, across which variance and autocorrelation stay finite, so that no such precursor can exist there. That result and this one concern different system classes: its defining feature, the model’s own output re-entering its input, is absent from supervised fine-tuning on a fixed external corpus. The paper states as an explicitly untested conjecture that the signature should fail on recursive self-training, and names the experiment that would decide it. The monitor’s statistical construction is inherited rather than proposed: it is a control chart with a baseline-relative limit, the Phase-I / Phase-II practice of statistical process control and change detection. What is specific here is the observable — a friction quantity read from the model’s own output distribution rather than a task metric or an internal state — and the honest limits that follow: the threshold is set to a ceiling on benign drift rather than to a false-alarm rate, and a panel of never-collapsing runs is named as the validation that would supply the guarantee sequential methods provide. Target venue: Transactions on Machine Learning Research. Data, notebooks, and analysis scripts are released with the paper. Part of the Friction Theory research programme; companion papers are linked in the related identifiers. New in version 2. The empirical results are unchanged; the revision is one of positioning. Version 1 shipped without a related-work section. This version adds one covering six literatures, narrows the novelty claim from the early-warning phenomenon (which prior work established) to the observable (that the precursor is legible in the output distribution, hence deployable without weight access), and reconciles the regime-map against its incumbents. It also positions the monitor against the statistical field that owns its construction — sequential change detection and control charts — conceding that the construction is inherited and locating the specific contribution in the observable. An estimator ablation on the existing trajectories, added here, reports the distributional moments as buying transfer across families rather than a longer lead. Reviewed by external dual peer-style review of the revision. Builds on v1.

Authors

Institutions

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-07-23
DOI
https://doi.org/10.5281/zenodo.21509897
Primary Topic
Parallel Computing and Optimization Techniques
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
preprint

Reading the substrate, not the score: a self-calibrating friction signature that warns of fine-tuning over-fitting and maps when interventions help

Tomas Pødenphant Lund
Zenodo (CERN European Organization for Nuclear Research)
Parallel Computing and Optimization Techniques
preprint

Reading the substrate, not the score: a self-calibrating friction signature that warns of fine-tuning over-fitting and maps when interventions help

Tomas Pødenphant Lund
preprint en

Abstract

Early warning of a training-time collapse has so far been read from a model’s internals. This paper shows the same precursor is legible in the model’s own output distribution: per-token entropy variance and lag-1 autocorrelation, computed from the log-probabilities any inference interface already returns, rise before held-out accuracy drops when narrow fine-tuning begins to erode general reasoning. The two precursors replicate across three model families (Qwen2.5-1.5B, Phi-3.5-mini, SmolLM2-1.7B) with multi-seed error bars, and the lead over the accuracy drop runs from several optimizer steps to coincident depending on the model. The signature is packaged as a self-calibrating monitor whose trigger is relative to each run’s own opening baseline, so there is no global threshold and no separate calibration pass. It flags the collapse in three of three families that collapse, while a difficulty-masking gate correctly skips a fourth. The load-bearing contrast is that the failure is capability-erosion rather than over-fit to the training task: the in-task validation loss falls straight through the collapse and never warns, and a ceiling-masked accuracy probe never moves, while the friction signature rises through the transition. An estimator ablation shows the advantage of the distributional moments over a scalar entropy level is transfer rather than lead — the level matches them on two families and fails on the third. Read across model capacity instead of training time, the same signature yields a regime-map that predicts when an instruction frame helps: it bites on pressed substrates and does nothing on frozen ones. A published account of entropy collapse proves it a first-order transition for feedback-amplified systems, across which variance and autocorrelation stay finite, so that no such precursor can exist there. That result and this one concern different system classes: its defining feature, the model’s own output re-entering its input, is absent from supervised fine-tuning on a fixed external corpus. The paper states as an explicitly untested conjecture that the signature should fail on recursive self-training, and names the experiment that would decide it. The monitor’s statistical construction is inherited rather than proposed: it is a control chart with a baseline-relative limit, the Phase-I / Phase-II practice of statistical process control and change detection. What is specific here is the observable — a friction quantity read from the model’s own output distribution rather than a task metric or an internal state — and the honest limits that follow: the threshold is set to a ceiling on benign drift rather than to a false-alarm rate, and a panel of never-collapsing runs is named as the validation that would supply the guarantee sequential methods provide. Target venue: Transactions on Machine Learning Research. Data, notebooks, and analysis scripts are released with the paper. Part of the Friction Theory research programme; companion papers are linked in the related identifiers. New in version 2. The empirical results are unchanged; the revision is one of positioning. Version 1 shipped without a related-work section. This version adds one covering six literatures, narrows the novelty claim from the early-warning phenomenon (which prior work established) to the observable (that the precursor is legible in the output distribution, hence deployable without weight access), and reconciles the regime-map against its incumbents. It also positions the monitor against the statistical field that owns its construction — sequential change detection and control charts — conceding that the construction is inherited and locating the specific contribution in the observable. An estimator ablation on the existing trajectories, added here, reports the distributional moments as buying transfer across families rather than a longer lead. Reviewed by external dual peer-style review of the revision. Builds on v1.

Zenodo (CERN European Organization for Nuclear Research)
Aarhus University (DK)
Parallel Computing and Optimization Techniques
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.