The Defaults That Train the World

Nearly every neural network trained on Earth is trained by numbers nobody derived. The step size, the two memory rates, and the small constant that keeps the division safe in the Adam optimizer; the fraction of units silenced by dropout; the rate at which batch normalization absorbs the statistics of the data — these numbers steer billions of training runs, and every one of them entered the world the same way: a researcher tried some settings on the problems in front of them, found settings that worked, and printed them as good defaults. That is an honest way to propose a number. It is not a derivation, and the field has spent the numbers ever since as if it were one. This paper takes a census. Row one: Adam's proposed defaults — a step size of 0.001, decay rates of 0.9 and 0.999, and an epsilon of 10⁻⁸ — offered by Kingma and Ba in a single sentence as "good default settings for the tested machine learning problems," with no derivation attached. Row two is the census's sharpest entry: the same named algorithm leaves the factory with two different epsilons tenfold apart — TensorFlow and Keras ship 10⁻⁷, PyTorch ships 10⁻⁸ — and epsilon is not inert packing material; where the gradient scale falls to epsilon's neighborhood, the two factory settings give steps that differ more than fivefold. Row three: dropout's retention probability of 0.5, printed by its authors as what "seems to be close to optimal for a wide range of networks and tasks." Row four: batch normalization's momentum, where the two leading frameworks use the same word for opposite seats in the update — and their defaults, 0.1 and 0.99, put weights of 0.1 and 0.01 on each new batch of data, a tenfold difference in how fast the running statistics learn, and a standing trap for anyone carrying a number across the border. My verdict is printed at full size, and it is a verdict about status, not quality: these are good defaults — that is exactly why they survived — but they are chosen, not derived, and the world's training runs on inherited settings. Grade: **CHOSEN, NOT DERIVED — THE WORLD'S TRAINING RUNS ON INHERITED SETTINGS, ONE OF THEM SPLIT TENFOLD.** This paper is part of a breadth program ranging beyond my earlier work; it touches no claim of the physics core.