Atomic Autoresearch: Autonomous Hyperparameter Optimization in a Browser-Based Scalar Autograd Transformer

This work ports Karpathy's autoresearch framework to a dependency-free, browser-executable implementation: a single 38 KB HTML file containing a complete GPT transformer, scalar autograd engine, Adam optimizer, evaluation harness, and autonomous experiment controller. The system implements architectural features from the original H100-scale codebase — Rotary Position Embeddings (RoPE), ReluSquared activations, value embeddings with gated mixing, logit soft-capping, and residual skip connections — each as a toggleable feature the search can enable or disable. Evaluation spans seven character-level sequence domains: English names (32K and 200 documents), antimicrobial peptides, SMILES molecular notation, startup company names, dinosaur names, and cocktail names. The autonomous loop discovers domain-dependent architectural preferences: data-rich domains benefit from feature additions, while data-poor domains benefit primarily from learning rate tuning. RoPE emerges as the most broadly beneficial feature (kept in 4 of 7 domains). Improvements range from 0.2% to 41.7% BPB reduction, achieved through up to 19 automated experiments per domain with zero human intervention. The system executes in any modern browser on commodity hardware with no perceptible lag, demonstrating that the autoresearch paradigm remains viable at the absolute minimum scale.