brAIAIn: A Language Model Trained Exclusively on What It Was Paid to Learn

Frontier language models are trained on trillions of tokens whose origin is rarely disclosed. We report on brAIAIn, a word trigram language model whose entire trainingset is public, numbered, and, with a small number of labeled exceptions, paid for. The first version was trained on a twelve-line founding text. Each later version adds exactly one sentence, either purchased by a member of the public for five US dollars or written by the lab and labeled as an unpaid internal release. Each version is also trained on passages written by its predecessor, a deliberate small-scale instance of recursive selftraining. Across 7 versions the vocabulary grew from 96 to 130 words, the share of training tokens written by the model itself rose from 0% to 41.9%, and the model scored 0.0% on all 1,319 GSM8K test problems in every version. On TruthfulQA MC1 its best perword score, 36.7%, falls just short of a baseline that always picks the longest answer (36.8%). We release every version's weights, the complete training data and the evaluation code under CC0.