SuperBake: Installing Verified Facts into Transformer Weights by Direct Construction

We present SuperBake, a system that installs new factual knowledge into the weights of large language models without any gradient steps. Instead of fine-tuning, low-rank adaptation, or retrieval, SuperBake measures how the stock model represents each question and then hand- constructs a small circuit of neurons — exact weights, written directly — in a region appended to the network’s MLPs. Each installed fact receives a physical address (layer and slot coordi- nates), is behaviorally verified across phrasing variants, and is neutralized (zeroed) if verification fails, so delivered weights never carry silently broken knowledge. The output is a standard check- point that stock inference code loads unchanged, together with a receipt listing every fact, its verification status, and its coordinates. On a 7,450-row battery over 1,000 facts (Llama-3.1-8B-Instruct), the constructed engine veri- fies 91.3% of rows overall against 76.9% for a strong masked-SGD baseline — while leaving prose perplexity at the stock baseline, where the SGD baseline more than doubles it. Constructed facts answer reverse questions at∼94% where the trained baseline scores 0.7%: the reversal curse does not apply when the reverse mapping is simply written. Delivered, stock-loadable checkpoints reach 93–97% primary-question recall on Llama-3.1-8B and Qwen2.5-7B. Construc- tion is fast: measured end-to-end bakes run in minutes on one 80 GB GPU, and a reduced pipeline wrote 192 facts into a raw Pythia-6.9B at 14.8 facts per second. The method was derived from a mechanistic post-mortem of what stochastic gradient descent actually builds when it crams facts into appended neurons. We report those findings: dense fact storage lands in a coherent sub-threshold population code living in activation-function leakage — the same signal that damages prose — and several transport and capacity laws that any weight-space editor must respect. SuperBake is the constructive rebuild of that mechanism: the same associative memory, but gated, addressable, verifiable, and harmless to the host model.