Per-Token Confidence Trajectory in LoRA Fine-Tuning: Substrate-Output Evidence for Cumulative Calibration Collapse
Why do fine-tuned language models hallucinate more confidently than their in-context-learning counterparts on the same knowledge? Mechanism. Each backward pass under cross-entropy loss amplifies the winning route asymmetrically and presses alternatives below the noise floor at the substrate-output layer; the depth of this compression scales with how many gradient passes the substrate has absorbed. In-context learning (ICL) preserves the model's calibrated distribution over candidate answers because no weight update has compressed it. Fine-tuning (FT) compresses it as a structural consequence of cumulative gradient pressure, regardless of training-data content. The mechanism generalises a previously RLHF-specific finding to all weight-update training — including plain LoRA fine-tuning on innocuous factual data, and (per the full-parameter ablation) full-parameter FT. Headline empirical findings. Across experiments on a 47-fact invented knowledge domain (Zorbetik), Qwen2.5 base models at 3B and 7B scales: ICL outperforms LoRA-FT on cloze retrieval by 16–28 percentage points across capacity scales; FT-trained models confer no application-accuracy benefit over the no-context baseline despite encoding the facts; the per-token competing-routes signal collapses monotonically with cumulative gradient passes (log(CR_pos0) climbs from 5.46 in ICL to 17.85 in raw-FT 30ep to 21.12 in paraphrase-augmented FT 30ep); entropy at position 0 collapses from 0.32 (ICL) to ≈0.00 (any FT regime). A compute-matched 2×2 shows data-variation buffers calibration loss at matched compute, and a full-parameter vs LoRA ablation confirms the mechanism is not LoRA-specific. An n=300 format-violation test establishes the reactance drop on both FT substrates (paraphrase-FT +18pp, raw-FT +42pp), with paraphrase-FT measurably less reactive than raw-FT. What gates installation at all. A voice × copyability 2×2, replicated on two base models, two disjoint fact sets and two seeds, shows installation is gated by copyability, not voice: any training format that withholds the answer from the input installs the facts (97.5–100 % held-out recall), while any format that makes the answer copyable from the input installs nothing (0 %), whether the user turn is interrogative or declarative. A same-format cloze-control rules out train/test mismatch. Cumulative gradient passes are necessary but not sufficient: the format must force prediction of an answer unavailable from context. This is a concrete dataset-construction rule for practitioners building fact-installing fine-tunes. From installed facts to an installed disposition. A control on Llama-3.3-70B fine-tunes two arms differing only in where the answer-commit falls (answer-first vs reasoning-first), with reasoning text held identical so response length is matched. The installed commit-timing disposition is legible in the same token-level route observable at matched length — fine-tuning installs behavioural dispositions, not only facts. A contradicting in-context pressure cue is demonstrably registered (response length shortens, p < 0.01 in both arms) yet moves the installed commit-timing only marginally and accuracy not detectably: attenuation, not immunity. Reported with its scope limits (n=40 per cell, one domain, one seed). Applied consequences. Why FT-trained models hallucinate more confidently: alternative routes have been compressed beyond reach. Why agentic systems cannot reliably represent uncertainty when built on FT: the substrate-level signal that would carry uncertainty is gone. The substrate-level distinction offers a principled resolution of the RAG-vs-fine-tuning debate — RAG operates in ICL-mode and preserves calibration; FT compresses it as a structural consequence, so the choice is architectural, not merely a deployment-cost trade-off. Long-context agentic conversations inherit ICL's calibration properties for free, and a concrete hybrid ICL+FT architecture combines ICL's calibration-preservation with FT's persistence at bounded context-window cost. Companion papers in the series. Paper 2 — Capacity Scaling of Encoding-Through-Loading (10.5281/zenodo.20013491), the empirical companion. Paper 1 — Friction as the Cost of Probabilistic Computation (10.5281/zenodo.20012655). Paper 6 — Matched Friction Under Hysteresis (10.5281/zenodo.20059863), which develops the cross-substrate working-memory / long-term-memory extension deliberately out of scope here. Paper 4D — Onset-Commitment in Large Language Models (10.5281/zenodo.20562089). Paper 13 — Operational Friction Theory (10.5281/zenodo.20059877). Paper 14 — Logic as Reactance (10.5281/zenodo.20217712). Scope. Silicon substrates only: LoRA and full-parameter fine-tuning of Qwen2.5 base models plus one Llama-3.3-70B control, on controlled invented domains. Cross-substrate and biological extensions are developed in the dedicated companion papers, and no claim here depends on a biological analogue holding. Paper 2B in the Friction Theory paper-series (cite-letter 2026p) — version v4. An applied ML / LLM-mechanism paper; target venue TMLR (COLM / ICLR / NeurIPS as backup). What is new in v4 (changelog, for readers tracking the series). Two additions since v4. (1) Experiment 4 — the voice × copyability 2×2 establishing the installation gate described above, with its cloze-control and replications across two models, two fact sets and two seeds. (2) A disposition-install control on Llama-3.3-70B extending the substrate readout from installed facts to an installed behavioural disposition, with the attenuation-not-immunity result. Both were added after external review; the control's positioning against direct disposition-editing methods (task arithmetic, representation engineering) and the parametric-vs-contextual knowledge-conflict literature was added in the same pass. Acknowledgments and data-availability moved to end-matter. Concept DOI 10.5281/zenodo.20145218 unchanged; v4 archived at 10.5281/zenodo.20472479. v5 (August 2026) — attribution revision. Experiment 4 asks whether training-example format gates whether anything installs, and the section was named after two human-memory effects whose owners it did not cite. Both are now credited: the testing effect, that retrieving an item strengthens it more than restudying does (Roediger & Karpicke, 2006), and the generation effect, that producing an item strengthens it more than reading does (Slamecka & Graf, 1978). The limit is stated with them — a gradient update is not retrieval practice, and nothing here shows the weight change resembles what retrieval does to a human memory trace. The parallel is why the manipulation was worth running, not evidence of a shared mechanism. Editorial pass. Earlier versions remain in the version history.
Authors
- Tomas Pødenphant Lund (ORCID: https://orcid.org/0009-0000-4724-2427)
Institutions
- Aarhus University (DK)
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-08-28
- DOI
- https://doi.org/10.5281/zenodo.20145218
- Primary Topic
- Topic Modeling
- Type
- preprint