Effective training pressure gates recursive knowledge degradation in LLMs: A multi-axis dose–response study
Recursive fine-tuning on synthetic data progressively degrades factual knowledge in large language models under common parameter-efficient settings. The conditions governing whether degradation emerges, remains bounded, or can be reversed are not well understood. Through dose–response experiments across three architectures (Qwen 2.5 1.5B, Gemma 3 1B, and Gemma 4 E2B IT), up to ten recursive generations, and three to six seeds per headline condition (Gemma 4 single-seed and therefore exploratory), we map adapter rank, learning rate, and synthetic exposure onto a regime structure with three zones: homeostatic, bounded, and degradative. Increasing rank produces a threshold-like transition within the tested grid; the boundary falls between effective ranks 50 and 88 for Qwen and between 3 and 6 for Gemma 3, a tenfold difference between architectures, and not the same transition type. Learning-rate sweeps under full fine-tuning provide exploratory evidence broadly consistent with the same qualitative pattern, consistent with perturbation magnitude as the operative variable rather than low-rank adaptation specifically. At intermediate capacity ( r = 128 , Qwen), factual retention remains near 90% while output-distribution quality collapses threefold, revealing a dissociation invisible to scalar retention metrics. Pre-registered dose–response experiments show that reducing synthetic exposure by 50% shifts the Qwen system toward the bounded regime (+9.2 pp at Gen 10, 95% CI [ 6.9,11.6 ] , p < 0.001 ) and improves endpoint retention on Gemma 3 (+12.9 pp, p = 0.003 ) without reducing the rate of loss. The effective-training-pressure index is an exploratory post-training summary, not validated as a prospective predictor across architectures or datasets. Experiments use 1–2 billion parameter models; generalization to larger scales is not established.
Authors
- Carlos Renato Lisboa Francês (ORCID: https://orcid.org/0000-0003-0305-7662)
- Carlos André de Mattos Teixeira (ORCID: https://orcid.org/0000-0003-1902-4347)
- Julio Leite Azancort Neto (ORCID: https://orcid.org/0000-0003-2866-5445)
- Carlos Ponce de León (ORCID: https://orcid.org/0000-0002-1907-5913)
Institutions
- Universidade de São Paulo (BR)
- Universidade Federal do Pará (BR)
Publication Details
- Journal
- Knowledge-Based Systems
- Published
- 2026-10-28
- DOI
- https://doi.org/10.1016/j.knosys.2026.117230
- Primary Topic
- Topic Modeling
- Type
- article
- Field-Weighted Citation Impact
- 0.00
Funders
- Coordenação de Aperfeiçoamento de Pessoal de Nível Superior
- Conselho Nacional de Desenvolvimento Científico e Tecnológico