Per-Token Confidence Trajectory in LoRA Fine-Tuning: Substrate-Output Evidence for Cumulative Calibration Collapse

Why do fine-tuned language models hallucinate more confidently than their in-context-learning counterparts on the same knowledge? Mechanism. Each backward pass under cross-entropy loss amplifies the winning route asymmetrically and presses alternatives below the noise floor at the substrate-output layer; the depth of this compression scales with how many gradient passes the substrate has absorbed. In-context learning (ICL) preserves the model's calibrated distribution over candidate answers because no weight update has compressed it. Fine-tuning (FT) compresses it as a structural consequence of cumulative gradient pressure, regardless of training-data content. The mechanism generalises a previously RLHF-specific finding to all weight-update training — including plain LoRA fine-tuning on innocuous factual data, and (per the full-parameter ablation) full-parameter FT. Headline empirical findings. Across experiments on a 47-fact invented knowledge domain (Zorbetik), Qwen2.5 base models at 3B and 7B scales: ICL outperforms LoRA-FT on cloze retrieval by 16–28 percentage points across capacity scales; FT-trained models confer no application-accuracy benefit over the no-context baseline despite encoding the facts; the per-token competing-routes signal collapses monotonically with cumulative gradient passes (log(CR_pos0) climbs from 5.46 in ICL to 17.85 in raw-FT 30ep to 21.12 in paraphrase-augmented FT 30ep); entropy at position 0 collapses from 0.32 (ICL) to ≈0.00 (any FT regime). A compute-matched 2×2 shows data-variation buffers calibration loss at matched compute, and a full-parameter vs LoRA ablation confirms the mechanism is not LoRA-specific. An n=300 format-violation test establishes the reactance drop on both FT substrates (paraphrase-FT +18pp, raw-FT +42pp), with paraphrase-FT measurably less reactive than raw-FT. What gates installation at all. A voice × copyability 2×2, replicated on two base models, two disjoint fact sets and two seeds, shows installation is gated by copyability, not voice: any training format that withholds the answer from the input installs the facts (97.5–100 % held-out recall), while any format that makes the answer copyable from the input installs nothing (0 %), whether the user turn is interrogative or declarative. A same-format cloze-control rules out train/test mismatch. Cumulative gradient passes are necessary but not sufficient: the format must force prediction of an answer unavailable from context. This is a concrete dataset-construction rule for practitioners building fact-installing fine-tunes. From installed facts to an installed disposition. A control on Llama-3.3-70B fine-tunes two arms differing only in where the answer-commit falls (answer-first vs reasoning-first), with reasoning text held identical so response length is matched. The installed commit-timing disposition is legible in the same token-level route observable at matched length — fine-tuning installs behavioural dispositions, not only facts. A contradicting in-context pressure cue is demonstrably registered (response length shortens, p < 0.01 in both arms) yet moves the installed commit-timing only marginally and accuracy not detectably: attenuation, not immunity. Reported with its scope limits (n=40 per cell, one domain, one seed). Applied consequences. Why FT-trained models hallucinate more confidently: alternative routes have been compressed beyond reach. Why agentic systems cannot reliably represent uncertainty when built on FT: the substrate-level signal that would carry uncertainty is gone. The substrate-level distinction offers a principled resolution of the RAG-vs-fine-tuning debate — RAG operates in ICL-mode and preserves calibration; FT compresses it as a structural consequence, so the choice is architectural, not merely a deployment-cost trade-off. Long-context agentic conversations inherit ICL's calibration properties for free, and a concrete hybrid ICL+FT architecture combines ICL's calibration-preservation with FT's persistence at bounded context-window cost. Companion papers in the series. Paper 2 — Capacity Scaling of Encoding-Through-Loading (10.5281/zenodo.20013491), the empirical companion. Paper 1 — Friction as the Cost of Probabilistic Computation (10.5281/zenodo.20012655). Paper 6 — Matched Friction Under Hysteresis (10.5281/zenodo.20059863), which develops the cross-substrate working-memory / long-term-memory extension deliberately out of scope here. Paper 4D — Onset-Commitment in Large Language Models (10.5281/zenodo.20562089). Paper 13 — Operational Friction Theory (10.5281/zenodo.20059877). Paper 14 — Logic as Reactance (10.5281/zenodo.20217712). Scope. Silicon substrates only: LoRA and full-parameter fine-tuning of Qwen2.5 base models plus one Llama-3.3-70B control, on controlled invented domains. Cross-substrate and biological extensions are developed in the dedicated companion papers, and no claim here depends on a biological analogue holding. Paper 2B in the Friction Theory paper-series (cite-letter 2026p) — version v4. An applied ML / LLM-mechanism paper; target venue TMLR (COLM / ICLR / NeurIPS as backup). What is new in v4 (changelog, for readers tracking the series). Two additions since v4. (1) Experiment 4 — the voice × copyability 2×2 establishing the installation gate described above, with its cloze-control and replications across two models, two fact sets and two seeds. (2) A disposition-install control on Llama-3.3-70B extending the substrate readout from installed facts to an installed behavioural disposition, with the attenuation-not-immunity result. Both were added after external review; the control's positioning against direct disposition-editing methods (task arithmetic, representation engineering) and the parametric-vs-contextual knowledge-conflict literature was added in the same pass. Acknowledgments and data-availability moved to end-matter. Concept DOI 10.5281/zenodo.20145218 unchanged; v4 archived at 10.5281/zenodo.20472479.

Authors

Institutions

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-07-23
DOI
https://doi.org/10.5281/zenodo.21520555
Primary Topic
Topic Modeling
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
preprint

Per-Token Confidence Trajectory in LoRA Fine-Tuning: Substrate-Output Evidence for Cumulative Calibration Collapse

Tomas Pødenphant Lund
Zenodo (CERN European Organization for Nuclear Research)
Topic Modeling
preprint

Per-Token Confidence Trajectory in LoRA Fine-Tuning: Substrate-Output Evidence for Cumulative Calibration Collapse

Tomas Pødenphant Lund
preprint en

Abstract

Why do fine-tuned language models hallucinate more confidently than their in-context-learning counterparts on the same knowledge? Mechanism. Each backward pass under cross-entropy loss amplifies the winning route asymmetrically and presses alternatives below the noise floor at the substrate-output layer; the depth of this compression scales with how many gradient passes the substrate has absorbed. In-context learning (ICL) preserves the model's calibrated distribution over candidate answers because no weight update has compressed it. Fine-tuning (FT) compresses it as a structural consequence of cumulative gradient pressure, regardless of training-data content. The mechanism generalises a previously RLHF-specific finding to all weight-update training — including plain LoRA fine-tuning on innocuous factual data, and (per the full-parameter ablation) full-parameter FT. Headline empirical findings. Across experiments on a 47-fact invented knowledge domain (Zorbetik), Qwen2.5 base models at 3B and 7B scales: ICL outperforms LoRA-FT on cloze retrieval by 16–28 percentage points across capacity scales; FT-trained models confer no application-accuracy benefit over the no-context baseline despite encoding the facts; the per-token competing-routes signal collapses monotonically with cumulative gradient passes (log(CR_pos0) climbs from 5.46 in ICL to 17.85 in raw-FT 30ep to 21.12 in paraphrase-augmented FT 30ep); entropy at position 0 collapses from 0.32 (ICL) to ≈0.00 (any FT regime). A compute-matched 2×2 shows data-variation buffers calibration loss at matched compute, and a full-parameter vs LoRA ablation confirms the mechanism is not LoRA-specific. An n=300 format-violation test establishes the reactance drop on both FT substrates (paraphrase-FT +18pp, raw-FT +42pp), with paraphrase-FT measurably less reactive than raw-FT. What gates installation at all. A voice × copyability 2×2, replicated on two base models, two disjoint fact sets and two seeds, shows installation is gated by copyability, not voice: any training format that withholds the answer from the input installs the facts (97.5–100 % held-out recall), while any format that makes the answer copyable from the input installs nothing (0 %), whether the user turn is interrogative or declarative. A same-format cloze-control rules out train/test mismatch. Cumulative gradient passes are necessary but not sufficient: the format must force prediction of an answer unavailable from context. This is a concrete dataset-construction rule for practitioners building fact-installing fine-tunes. From installed facts to an installed disposition. A control on Llama-3.3-70B fine-tunes two arms differing only in where the answer-commit falls (answer-first vs reasoning-first), with reasoning text held identical so response length is matched. The installed commit-timing disposition is legible in the same token-level route observable at matched length — fine-tuning installs behavioural dispositions, not only facts. A contradicting in-context pressure cue is demonstrably registered (response length shortens, p < 0.01 in both arms) yet moves the installed commit-timing only marginally and accuracy not detectably: attenuation, not immunity. Reported with its scope limits (n=40 per cell, one domain, one seed). Applied consequences. Why FT-trained models hallucinate more confidently: alternative routes have been compressed beyond reach. Why agentic systems cannot reliably represent uncertainty when built on FT: the substrate-level signal that would carry uncertainty is gone. The substrate-level distinction offers a principled resolution of the RAG-vs-fine-tuning debate — RAG operates in ICL-mode and preserves calibration; FT compresses it as a structural consequence, so the choice is architectural, not merely a deployment-cost trade-off. Long-context agentic conversations inherit ICL's calibration properties for free, and a concrete hybrid ICL+FT architecture combines ICL's calibration-preservation with FT's persistence at bounded context-window cost. Companion papers in the series. Paper 2 — Capacity Scaling of Encoding-Through-Loading (10.5281/zenodo.20013491), the empirical companion. Paper 1 — Friction as the Cost of Probabilistic Computation (10.5281/zenodo.20012655). Paper 6 — Matched Friction Under Hysteresis (10.5281/zenodo.20059863), which develops the cross-substrate working-memory / long-term-memory extension deliberately out of scope here. Paper 4D — Onset-Commitment in Large Language Models (10.5281/zenodo.20562089). Paper 13 — Operational Friction Theory (10.5281/zenodo.20059877). Paper 14 — Logic as Reactance (10.5281/zenodo.20217712). Scope. Silicon substrates only: LoRA and full-parameter fine-tuning of Qwen2.5 base models plus one Llama-3.3-70B control, on controlled invented domains. Cross-substrate and biological extensions are developed in the dedicated companion papers, and no claim here depends on a biological analogue holding. Paper 2B in the Friction Theory paper-series (cite-letter 2026p) — version v4. An applied ML / LLM-mechanism paper; target venue TMLR (COLM / ICLR / NeurIPS as backup). What is new in v4 (changelog, for readers tracking the series). Two additions since v4. (1) Experiment 4 — the voice × copyability 2×2 establishing the installation gate described above, with its cloze-control and replications across two models, two fact sets and two seeds. (2) A disposition-install control on Llama-3.3-70B extending the substrate readout from installed facts to an installed behavioural disposition, with the attenuation-not-immunity result. Both were added after external review; the control's positioning against direct disposition-editing methods (task arithmetic, representation engineering) and the parametric-vs-contextual knowledge-conflict literature was added in the same pass. Acknowledgments and data-availability moved to end-matter. Concept DOI 10.5281/zenodo.20145218 unchanged; v4 archived at 10.5281/zenodo.20472479.

Zenodo (CERN European Organization for Nuclear Research)
Aarhus University (DK)
Quality Education
Topic Modeling
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.