Capacity Scaling of Encoding-Through-Loading: Application vs. Cloze Asymmetry Across Three Orders of Magnitude
Abstract. Large language models solve two differentiable task types on the same underlying knowledge base. Cloze retrieval saturates early (~90 % by 8 B parameters); application scales monotonically across three orders of magnitude. We document this asymmetry on a single invented knowledge domain ("Zorbetik") across nine models, using frontloaded in-context learning (Brown et al. 2020). Four findings: (1) application scales monotonically with capacity (Spearman ρ = +1.000 on Qwen2.5; cross-family panel ρ = +0.92, n=9; +40.8 pp per decade); (2) the bottleneck migrates with capacity; (3) Mixture-of-Experts models scale on active parameters, not total; (4) at uniform n = 47 with Wilson 95 % CIs, four capable instruction-tuned substrates cluster in a capacity-independent 79-85 % application band, the empirical signature of a race-architecture floor (Paper 1 §2.5; Paper 10 §1.5). A companion paper (Paper 2C, in preparation) re-tests the floor on an independent 327-substance domain and on the chain-depth axis. Companion papers in the series: Paper 0 (BFT): 10.5281/zenodo.19462500 Paper 1 (FT generalised): 10.5281/zenodo.20012655 Paper 2B (substrate-mechanism companion, in preparation) Paper 2C (chain-depth axis companion, in preparation) Paper 3 (Friction-guided inference): 10.5281/zenodo.20014122 Paper 10 (Race-architecture, physics scope): 10.5281/zenodo.20014568 Data, fine-tuning notebooks, analysis scripts: https://github.com/tplund/friction-theory-p2-capacity-scaling v5 update (July 2026) — correction release. This version corrects the paper's central quantitative result. A full audit of the application benchmark, in which every answer key was independently re-derived from the 47-fact knowledge block, found that 7 of the 47 keys were wrong and 12 questions were ill-posed — 40 % of the application bank was broken — and that the scoring rubric was asymmetric between the two task types. Under corrected scoring, the dramatic capacity scaling reported in v1–v4 (application accuracy rising from 2 % to 85 %, Spearman ρ = +1.0) does not exist: it was an artifact of the broken keys, the asymmetric rubric, and the ill-posed items. The corrected data show a modest positive trend instead — approximately 60 % at 1.5B rising to an 80–86 % plateau for capable large models — with genuine derivation errors persisting at the top of the ladder. All archived model responses were re-scored against the corrected key on the 35 valid items, using format-tolerant matching plus manual adjudication of every non-match; the key re-derivation and the adjudications were independently validated blind by a separate model family. The audit methodology is reported as a first-class contribution, and the paper now credits the established literatures on benchmark data contamination, synthetic-knowledge probing, and pervasive benchmark label errors. The quantitative claims of v1–v4 are superseded; earlier versions remain accessible through the version history.
Institutions
- Aarhus University (DK)
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-07-24
- DOI
- https://doi.org/10.5281/zenodo.21519700
- Primary Topic
- Machine Learning in Materials Science
- Type
- preprint