Per-Token Confidence Trajectory in LoRA Fine-Tuning: Substrate-Output Evidence for Cumulative Calibration Collapse

Why do fine-tuned language models hallucinate more confidently than their in-context-learning counterparts on the same knowledge? Mechanism. Each backward pass under cross-entropy loss amplifies the winning route asymmetrically and presses alternatives below the noise floor at the substrate-output layer; the depth of this compression scales with how many gradient passes the substrate has absorbed. In-context learning (ICL) preserves the model's calibrated distribution over candidate answers because no weight update has compressed it. Fine-tuning (FT) compresses it as a structural consequence of cumulative gradient pressure, regardless of training-data content. The mechanism generalises a previously RLHF-specific finding to all weight-update training — including plain LoRA fine-tuning on innocuous factual data, and (per the full-parameter ablation) full-parameter FT. Headline empirical findings. Across experiments on a 47-fact invented knowledge domain (Zorbetik), Qwen2.5 base models at 3B and 7B scales: ICL outperforms LoRA-FT on cloze retrieval by 16–28 percentage points across capacity scales; FT-trained models confer no application-accuracy benefit over the no-context baseline despite encoding the facts; the per-token competing-routes signal collapses monotonically with cumulative gradient passes (log(CR_pos0) climbs from 5.46 in ICL to 17.85 in raw-FT 30ep to 21.12 in paraphrase-augmented FT 30ep); entropy at position 0 collapses from 0.32 (ICL) to ≈0.00 (any FT regime). A compute-matched 2×2 shows data-variation buffers calibration loss at matched compute, and a full-parameter vs LoRA ablation confirms the mechanism is not LoRA-specific. An n=300 format-violation test establishes the reactance drop on both FT substrates (paraphrase-FT +18pp, raw-FT +42pp), with paraphrase-FT measurably less reactive than raw-FT. What gates installation at all. A voice × copyability 2×2, replicated on two base models, two disjoint fact sets and two seeds, shows installation is gated by copyability, not voice: any training format that withholds the answer from the input installs the facts (97.5–100 % held-out recall), while any format that makes the answer copyable from the input installs nothing (0 %), whether the user turn is interrogative or declarative. A same-format cloze-control rules out train/test mismatch. Cumulative gradient passes are necessary but not sufficient: the format must force prediction of an answer unavailable from context. This is a concrete dataset-construction rule for practitioners building fact-installing fine-tunes. From installed facts to an installed disposition. A control on Llama-3.3-70B fine-tunes two arms differing only in where the answer-commit falls (answer-first vs reasoning-first), with reasoning text held identical so response length is matched. The installed commit-timing disposition is legible in the same token-level route observable at matched length — fine-tuning installs behavioural dispositions, not only facts. A contradicting in-context pressure cue is demonstrably registered (response length shortens, p < 0.01 in both arms) yet moves the installed commit-timing only marginally and accuracy not detectably: attenuation, not immunity. Reported with its scope limits (n=40 per cell, one domain, one seed). Applied consequences. Why FT-trained models hallucinate more confidently: alternative routes have been compressed beyond reach. Why agentic systems cannot reliably represent uncertainty when built on FT: the substrate-level signal that would carry uncertainty is gone. The substrate-level distinction offers a principled resolution of the RAG-vs-fine-tuning debate — RAG operates in ICL-mode and preserves calibration; FT compresses it as a structural consequence, so the choice is architectural, not merely a deployment-cost trade-off. Long-context agentic conversations inherit ICL's calibration properties for free, and a concrete hybrid ICL+FT architecture combines ICL's calibration-preservation with FT's persistence at bounded context-window cost. Companion papers in the series. Paper 2 — Capacity Scaling of Encoding-Through-Loading (10.5281/zenodo.20013491), the empirical companion. Paper 1 — Friction as the Cost of Probabilistic Computation (10.5281/zenodo.20012655). Paper 6 — Matched Friction Under Hysteresis (10.5281/zenodo.20059863), which develops the cross-substrate working-memory / long-term-memory extension deliberately out of scope here. Paper 4D — Onset-Commitment in Large Language Models (10.5281/zenodo.20562089). Paper 13 — Operational Friction Theory (10.5281/zenodo.20059877). Paper 14 — Logic as Reactance (10.5281/zenodo.20217712). Scope. Silicon substrates only: LoRA and full-parameter fine-tuning of Qwen2.5 base models plus one Llama-3.3-70B control, on controlled invented domains. Cross-substrate and biological extensions are developed in the dedicated companion papers, and no claim here depends on a biological analogue holding. Paper 2B in the Friction Theory paper-series (cite-letter 2026p) — version v4. An applied ML / LLM-mechanism paper; target venue TMLR (COLM / ICLR / NeurIPS as backup). What is new in v4 (changelog, for readers tracking the series). Two additions since v4. (1) Experiment 4 — the voice × copyability 2×2 establishing the installation gate described above, with its cloze-control and replications across two models, two fact sets and two seeds. (2) A disposition-install control on Llama-3.3-70B extending the substrate readout from installed facts to an installed behavioural disposition, with the attenuation-not-immunity result. Both were added after external review; the control's positioning against direct disposition-editing methods (task arithmetic, representation engineering) and the parametric-vs-contextual knowledge-conflict literature was added in the same pass. Acknowledgments and data-availability moved to end-matter. Concept DOI 10.5281/zenodo.20145218 unchanged; v4 archived at 10.5281/zenodo.20472479. v5 (August 2026) — attribution revision. Experiment 4 asks whether training-example format gates whether anything installs, and the section was named after two human-memory effects whose owners it did not cite. Both are now credited: the testing effect, that retrieving an item strengthens it more than restudying does (Roediger & Karpicke, 2006), and the generation effect, that producing an item strengthens it more than reading does (Slamecka & Graf, 1978). The limit is stated with them — a gradient update is not retrieval practice, and nothing here shows the weight change resembles what retrieval does to a human memory trace. The parallel is why the manipulation was worth running, not evidence of a shared mechanism. Editorial pass. Earlier versions remain in the version history.

Authors

Institutions

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-08-28
DOI
https://doi.org/10.5281/zenodo.20145218
Primary Topic
Topic Modeling
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
preprint

Per-Token Confidence Trajectory in LoRA Fine-Tuning: Substrate-Output Evidence for Cumulative Calibration Collapse

Tomas Pødenphant Lund
Zenodo (CERN European Organization for Nuclear Research)
Topic Modeling
preprint

Per-Token Confidence Trajectory in LoRA Fine-Tuning: Substrate-Output Evidence for Cumulative Calibration Collapse

Tomas Pødenphant Lund
preprint en

Abstract

Why do fine-tuned language models hallucinate more confidently than their in-context-learning counterparts on the same knowledge? Mechanism. Each backward pass under cross-entropy loss amplifies the winning route asymmetrically and presses alternatives below the noise floor at the substrate-output layer; the depth of this compression scales with how many gradient passes the substrate has absorbed. In-context learning (ICL) preserves the model's calibrated distribution over candidate answers because no weight update has compressed it. Fine-tuning (FT) compresses it as a structural consequence of cumulative gradient pressure, regardless of training-data content. The mechanism generalises a previously RLHF-specific finding to all weight-update training — including plain LoRA fine-tuning on innocuous factual data, and (per the full-parameter ablation) full-parameter FT. Headline empirical findings. Across experiments on a 47-fact invented knowledge domain (Zorbetik), Qwen2.5 base models at 3B and 7B scales: ICL outperforms LoRA-FT on cloze retrieval by 16–28 percentage points across capacity scales; FT-trained models confer no application-accuracy benefit over the no-context baseline despite encoding the facts; the per-token competing-routes signal collapses monotonically with cumulative gradient passes (log(CR_pos0) climbs from 5.46 in ICL to 17.85 in raw-FT 30ep to 21.12 in paraphrase-augmented FT 30ep); entropy at position 0 collapses from 0.32 (ICL) to ≈0.00 (any FT regime). A compute-matched 2×2 shows data-variation buffers calibration loss at matched compute, and a full-parameter vs LoRA ablation confirms the mechanism is not LoRA-specific. An n=300 format-violation test establishes the reactance drop on both FT substrates (paraphrase-FT +18pp, raw-FT +42pp), with paraphrase-FT measurably less reactive than raw-FT. What gates installation at all. A voice × copyability 2×2, replicated on two base models, two disjoint fact sets and two seeds, shows installation is gated by copyability, not voice: any training format that withholds the answer from the input installs the facts (97.5–100 % held-out recall), while any format that makes the answer copyable from the input installs nothing (0 %), whether the user turn is interrogative or declarative. A same-format cloze-control rules out train/test mismatch. Cumulative gradient passes are necessary but not sufficient: the format must force prediction of an answer unavailable from context. This is a concrete dataset-construction rule for practitioners building fact-installing fine-tunes. From installed facts to an installed disposition. A control on Llama-3.3-70B fine-tunes two arms differing only in where the answer-commit falls (answer-first vs reasoning-first), with reasoning text held identical so response length is matched. The installed commit-timing disposition is legible in the same token-level route observable at matched length — fine-tuning installs behavioural dispositions, not only facts. A contradicting in-context pressure cue is demonstrably registered (response length shortens, p < 0.01 in both arms) yet moves the installed commit-timing only marginally and accuracy not detectably: attenuation, not immunity. Reported with its scope limits (n=40 per cell, one domain, one seed). Applied consequences. Why FT-trained models hallucinate more confidently: alternative routes have been compressed beyond reach. Why agentic systems cannot reliably represent uncertainty when built on FT: the substrate-level signal that would carry uncertainty is gone. The substrate-level distinction offers a principled resolution of the RAG-vs-fine-tuning debate — RAG operates in ICL-mode and preserves calibration; FT compresses it as a structural consequence, so the choice is architectural, not merely a deployment-cost trade-off. Long-context agentic conversations inherit ICL's calibration properties for free, and a concrete hybrid ICL+FT architecture combines ICL's calibration-preservation with FT's persistence at bounded context-window cost. Companion papers in the series. Paper 2 — Capacity Scaling of Encoding-Through-Loading (10.5281/zenodo.20013491), the empirical companion. Paper 1 — Friction as the Cost of Probabilistic Computation (10.5281/zenodo.20012655). Paper 6 — Matched Friction Under Hysteresis (10.5281/zenodo.20059863), which develops the cross-substrate working-memory / long-term-memory extension deliberately out of scope here. Paper 4D — Onset-Commitment in Large Language Models (10.5281/zenodo.20562089). Paper 13 — Operational Friction Theory (10.5281/zenodo.20059877). Paper 14 — Logic as Reactance (10.5281/zenodo.20217712). Scope. Silicon substrates only: LoRA and full-parameter fine-tuning of Qwen2.5 base models plus one Llama-3.3-70B control, on controlled invented domains. Cross-substrate and biological extensions are developed in the dedicated companion papers, and no claim here depends on a biological analogue holding. Paper 2B in the Friction Theory paper-series (cite-letter 2026p) — version v4. An applied ML / LLM-mechanism paper; target venue TMLR (COLM / ICLR / NeurIPS as backup). What is new in v4 (changelog, for readers tracking the series). Two additions since v4. (1) Experiment 4 — the voice × copyability 2×2 establishing the installation gate described above, with its cloze-control and replications across two models, two fact sets and two seeds. (2) A disposition-install control on Llama-3.3-70B extending the substrate readout from installed facts to an installed behavioural disposition, with the attenuation-not-immunity result. Both were added after external review; the control's positioning against direct disposition-editing methods (task arithmetic, representation engineering) and the parametric-vs-contextual knowledge-conflict literature was added in the same pass. Acknowledgments and data-availability moved to end-matter. Concept DOI 10.5281/zenodo.20145218 unchanged; v4 archived at 10.5281/zenodo.20472479. v5 (August 2026) — attribution revision. Experiment 4 asks whether training-example format gates whether anything installs, and the section was named after two human-memory effects whose owners it did not cite. Both are now credited: the testing effect, that retrieving an item strengthens it more than restudying does (Roediger & Karpicke, 2006), and the generation effect, that producing an item strengthens it more than reading does (Slamecka & Graf, 1978). The limit is stated with them — a gradient update is not retrieval practice, and nothing here shows the weight change resembles what retrieval does to a human memory trace. The parallel is why the manipulation was worth running, not evidence of a shared mechanism. Editorial pass. Earlier versions remain in the version history.

Zenodo (CERN European Organization for Nuclear Research)
Aarhus University (DK)
Topic Modeling
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.