Template Compatibility Governs Behavioural Fine-Tuning at 70B: A Collapse Cliff, a Truncation Artifact, and When Doubt Pays
Template compatibility, not training-set size, governs whether a behavioural fine-tune survives. Fine-tuning a 70B model (Llama-3.3-70B) on a short premise-check template preserves general capability at the matched dose (full-benchmark MMLU 82.3% versus a vanilla 84.5%, n = 14,042), while a verbose template of the same size and dose collapses it to 8.2% — a cliff between 90 and 120 examples, not a slope. The collapse is behavioural: under a format-invariant forced-choice log-probability readout the collapsed weights still score 78.3% (versus vanilla 81.4%), so the incompatible template overwrites answer-delivery while leaving the stored knowledge in place. The identical recipe applied to an 8B model of the same family produces no cliff at all across three fine-tuning seeds per dose, so the damage is scale-gated rather than a property of the training configuration. That fine-tuning and in-context learning shape response behaviour more than stored knowledge is the superficial-alignment picture, established in prior work and actively contested there; no priority is claimed for the phenomenon. This paper takes the independently documented pattern as the warrant for a narrower, quantitative question — where the boundary bites and what it costs — and supplies three measurements: the template-compatibility cliff above; a cautionary methodological finding that an apparent +10 percentage-point in-context “accuracy lift” can be manufactured entirely by a short generation cap that truncates open reasoning before it commits (the gain is pure commit-rate, not reasoning); and a characterisation of in-context premise-checking as installing a task-dependent caution disposition rather than accuracy — it raises abstention and false-premise rejection without lifting accuracy on any substrate, its benefit saturates on overt false premises (aligned models already reject at 89–100%), and its cost of over-rejecting valid items is capacity-gated, for which a closed-form crossover is offered as a descriptive model of the trade-off. Data and code. The training pools, in-context exemplar sets, the false/valid-premise evaluation set, the deterministic scorer, the LoRA adapters, and the raw generations are released with the paper. v2 update (July 2026). The empirical results are unchanged. This version corrects the paper's positioning against prior work. The disposition-versus-knowledge boundary it measures is the superficial-alignment hypothesis, established well before this work and contested there; v2 credits that literature in full, claims no priority for the phenomenon, and reframes the contribution as the measurements rather than the boundary — and the title is revised accordingly. A new related-work discussion engages the superficial-alignment, catastrophic-forgetting, abstention, and emergent-misalignment literatures; a scale control (an 8B model of the same family shows no cliff) is added to bound the single-recipe explanation; and the abstention saturation claim is scoped to overt false premises as an explicit hypothesis. Builds on v1. v3 (August 2026) — attribution revision. The collapse this paper measures is met from the other side by catastrophic interference (McCloskey & Cohen, 1989; French, 1999), and section 1.5 now says so: no new mechanism for the loss is claimed, only a threshold located in a variable that literature does not manipulate. Section 5's closed-form crossover in false-premise density is placed as what it is — a break-even under uncertainty, whose position given a prior and a payoff matrix is criterion placement (Green & Swets, 1966) — with the increment kept: the two inputs are measured here rather than assumed. Editorial pass. Earlier versions remain in the version history.
Authors
- Tomas Pødenphant Lund (ORCID: https://orcid.org/0009-0000-4724-2427)
Institutions
- Aarhus University (DK)
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-08-28
- DOI
- https://doi.org/10.5281/zenodo.20562086
- Primary Topic
- Natural Language Processing Techniques
- Type
- preprint