Banya AI 1: A Language-Model Training Engine Using Hand-Derived Closed-Form Exact Adjoints Without Automatic Differentiation

This paper presents an engine that trains a language model without automatic differentiation. For every operation primitive that makes up the network, the engine directly implements the Jacobian transpose (the adjoint) as a hand-derived closed-form expression. The gradients computed this way are numerically identical to standard backpropagation, yet no automatic-differentiation graph is ever built. In our measurements, no automatic-differentiation library was loaded into memory; the normalization scale was constant from 2 to 10 blocks; and the median ratio of the hand-derived adjoint to finite differences was 0.9997. In a side-by-side training comparison with the same small architecture, initial weights, and batch sequence run for 300 steps against a standard-backpropagation reference implementation, the loss trajectories differed by less than 1e-06 over the whole run, and the final loss was identical at 1.1039. We further quantify the confirmed gains: operations that are hard to hook exactly into a training loop under automatic differentiation (circulant-matrix channel mixing, operations without a partition function) can be trained with exact adjoints; circulant channel mixing reduces parameters by roughly a factor of 1024 relative to a dense matrix; and block-axis batched execution was up to 5.07 times faster than sequential execution on mixing operations. The core claim is not "we do not use backpropagation" but "what becomes possible, and what becomes cheap, when automatic differentiation is replaced by hand-derived exact adjoints." This record contains the English edition (_en.pdf) and the Korean original (_kr.pdf). Part 1 of the six-part Banya AI research series. All measured figures are reproducible with the public code package: https://ubmscoin.github.io/banya/banya_ai_paper_code/ (code Apache-2.0). Frozen model checkpoints: https://doi.org/10.5281/zenodo.21383724