Effective training pressure gates recursive knowledge degradation in LLMs: A multi-axis dose–response study

Recursive fine-tuning on synthetic data progressively degrades factual knowledge in large language models under common parameter-efficient settings. The conditions governing whether degradation emerges, remains bounded, or can be reversed are not well understood. Through dose–response experiments across three architectures (Qwen 2.5 1.5B, Gemma 3 1B, and Gemma 4 E2B IT), up to ten recursive generations, and three to six seeds per headline condition (Gemma 4 single-seed and therefore exploratory), we map adapter rank, learning rate, and synthetic exposure onto a regime structure with three zones: homeostatic, bounded, and degradative. Increasing rank produces a threshold-like transition within the tested grid; the boundary falls between effective ranks 50 and 88 for Qwen and between 3 and 6 for Gemma 3, a tenfold difference between architectures, and not the same transition type. Learning-rate sweeps under full fine-tuning provide exploratory evidence broadly consistent with the same qualitative pattern, consistent with perturbation magnitude as the operative variable rather than low-rank adaptation specifically. At intermediate capacity ( r = 128 , Qwen), factual retention remains near 90% while output-distribution quality collapses threefold, revealing a dissociation invisible to scalar retention metrics. Pre-registered dose–response experiments show that reducing synthetic exposure by 50% shifts the Qwen system toward the bounded regime (+9.2 pp at Gen 10, 95% CI [ 6.9,11.6 ] , p < 0.001 ) and improves endpoint retention on Gemma 3 (+12.9 pp, p = 0.003 ) without reducing the rate of loss. The effective-training-pressure index is an exploratory post-training summary, not validated as a prospective predictor across architectures or datasets. Experiments use 1–2 billion parameter models; generalization to larger scales is not established.

Authors

Institutions

Publication Details

Journal
Knowledge-Based Systems
Published
2026-10-28
DOI
https://doi.org/10.1016/j.knosys.2026.117230
Primary Topic
Topic Modeling
Type
article
Field-Weighted Citation Impact
0.00

Funders

Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Effective training pressure gates recursive knowledge degradation in LLMs: A multi-axis dose–response study

Carlos Renato Lisboa Francês, Carlos André de Mattos Teixeira, Julio Leite Azancort Neto, Carlos Ponce de León
1 links
Knowledge-Based Systems
Topic Modeling
article

Effective training pressure gates recursive knowledge degradation in LLMs: A multi-axis dose–response study

Carlos Renato Lisboa Francês, Carlos André de Mattos Teixeira, Julio Leite Azancort Neto, Carlos Ponce de León
article en

Abstract

Recursive fine-tuning on synthetic data progressively degrades factual knowledge in large language models under common parameter-efficient settings. The conditions governing whether degradation emerges, remains bounded, or can be reversed are not well understood. Through dose–response experiments across three architectures (Qwen 2.5 1.5B, Gemma 3 1B, and Gemma 4 E2B IT), up to ten recursive generations, and three to six seeds per headline condition (Gemma 4 single-seed and therefore exploratory), we map adapter rank, learning rate, and synthetic exposure onto a regime structure with three zones: homeostatic, bounded, and degradative. Increasing rank produces a threshold-like transition within the tested grid; the boundary falls between effective ranks 50 and 88 for Qwen and between 3 and 6 for Gemma 3, a tenfold difference between architectures, and not the same transition type. Learning-rate sweeps under full fine-tuning provide exploratory evidence broadly consistent with the same qualitative pattern, consistent with perturbation magnitude as the operative variable rather than low-rank adaptation specifically. At intermediate capacity ( r = 128 , Qwen), factual retention remains near 90% while output-distribution quality collapses threefold, revealing a dissociation invisible to scalar retention metrics. Pre-registered dose–response experiments show that reducing synthetic exposure by 50% shifts the Qwen system toward the bounded regime (+9.2 pp at Gen 10, 95% CI [ 6.9,11.6 ] , p < 0.001 ) and improves endpoint retention on Gemma 3 (+12.9 pp, p = 0.003 ) without reducing the rate of loss. The effective-training-pressure index is an exploratory post-training summary, not validated as a prospective predictor across architectures or datasets. Experiments use 1–2 billion parameter models; generalization to larger scales is not established.

Knowledge-Based SystemsVol. 354
Universidade de São Paulo (BR), Universidade Federal do Pará (BR)
Coordenação de Aperfeiçoamento de Pessoal de Nível Superior, Conselho Nacional de Desenvolvimento Científico e Tecnológico
Openalex Percentile: Top 13%
Topic Modeling
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon