Twenty Tokens: The Fit That Moved

In 2022 a single ratio began steering how the world spends computing power on language models: train at about twenty tokens for every parameter. The ratio came from Hoffmann and collaborators' Chinchilla study, which trained over 400 models and estimated the compute-optimal frontier three different ways. Two of the three ways read the frontier nearly directly off the experiments, and both pointed near twenty to one — and the model the authors built to prove the point, a 70-billion-parameter network trained on 1.4 trillion tokens, sits at exactly twenty to one. The third way was different in kind: a five-constant formula for the loss, fitted to the runs, from which the frontier is computed rather than read. That formula's constants are the ones the field copied into its own papers and planning documents. In 2024, Besiroglu, Erdil, Barnett and You at Epoch AI attempted to replicate that third estimate, and the printed constants did not survive: evaluated honestly, they prescribe not twenty tokens per parameter but roughly seventy; they fit the very data they were extracted from badly; and the confidence intervals printed beside them are so narrow that earning them would have required over 600,000 experiments, against fewer than 500 actually run. The replication found the causes, and the original authors confirmed them: the fitting optimizer stopped before convergence, because the loss was averaged rather than summed and its scale defeated the stopping rule, and the constants as printed in the paper's body are rounded in a way that biases the formula's predictions — rounding the data exponent alone inflates one term of the law by about thirteen percent at a typical training size. Refitted carefully, the same formula on the same reconstructed data lands back near twenty tokens per parameter, in line with the other two approaches. My own notebook, spending only the constants exactly as the replication prints them, reproduces the split from the constants themselves: at Chinchilla's compute budget the printed constants prescribe 59 tokens per parameter in their precise source form and 93 in their rounded published form, while the refit prescribes 18. I fence this paper at full size in both directions: the direction of the law — grow parameters and tokens together — stands under all three approaches and under the refit, and this paper does not debunk scaling. What failed is narrower and, for an era that budgets billions by formula, serious enough: the printed constants of the parametric law, and the intervals printed beside them. Grade: **THE DIRECTION STANDS — THE PRINTED CONSTANTS DID NOT REPLICATE, PRICED.** This paper is part of a breadth program ranging beyond my earlier work; it touches no claim of the physics core.