The Missing Medium: A Transactional Residual Stream for Transformer Training

Modern training treats the transformer residual stream as an anonymous accumulator: any kernel may execute h += \delta, and no record of the write survives the step. Training observes the consequences of internal writes — loss curves, evaluations, activation probes — never the writes themselves. We introduce hytorch, a training loop in which the residual stream is a typed, transactional channel: blocks propose, a pinned policy admits, and every candidate write leaves exactly one 16-byte record — commit, overflow (lost a slot contention) or abort (rule rejection) — so that what the model attempted to write, and why it was refused, is data. Every optimizer step is gated by a durable receipt from an embedded transactional store (Hyphae); a CPU-only verifier replays audited microbatches bit-for-bit against a pinned software reference; the same reference binary admits NVIDIA (sm_89/sm_90), AMD (gfx950) and, through a two-phase policy, AWS Trainium2 kernels on 200,000 adversarial cases with bit-identical verdicts and residuals. Fault injection over real training spills is detected 36/36 with the correct class. The instrument produced two results. Our preregistered protocol reversed our own headline: a single-LR -44.9% perplexity win, replicated across vendors, became a 2.9× loss to the properly tuned dense twin under the preregistered LR sweep, and we publish the reversal. And at nanochat-d20 scale the ledger recorded a silent channel collapse: commit rate 0.066%→0 by step 200, the write codebook frozen bit-exactly, while validation loss improved on schedule through the architecture's declared bypass paths. Neither loss, evaluation nor activation probes distinguish an expensive pathway from a dead one; the write records do, mechanically, during training. We are explicit about what this is and is not. hytorch is an instrumentation and verification framework, not an efficient training architecture: the channel configuration we tested (at most 160 scalar degrees of freedom per token per layer) does not carry a language model — the cataloged d20 model is at chance on knowledge tasks — and costs 12–29× the twin's wall clock; its headline capacity number comes from a run whose STE gradient we later found to be mis-scaled, and the corrected rerun does not yet exist. What the evidence supports is narrower and, we argue, more useful: a training process whose state transitions through the typed channel are witnessed, receipted and replayable. Version 1.1 (5 September 2026) supersedes v1.0 (4 September 2026) after four external reviews; the review synthesis and the list of changes are in the repository (paper/notes/REVIEWS-2026-09.md). Code and evidence: https://github.com/Hyphae-Research-Foundation/hytorch (release v1.1). Source code is licensed Apache-2.0; this paper, its figures and the results prose CC BY-SA 4.0. Every number in the paper traces to a file under results/ or a record key in a Hyphae ledger.

Authors

Institutions

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-09-05
DOI
https://doi.org/10.5281/zenodo.22059492
Primary Topic
Security and Verification in Computing
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
preprint

The Missing Medium: A Transactional Residual Stream for Transformer Training

Hyphae Research Foundation, Mario Gutierrez
Zenodo (CERN European Organization for Nuclear Research)
Security and Verification in Computing
preprint

The Missing Medium: A Transactional Residual Stream for Transformer Training

Hyphae Research Foundation, Mario Gutierrez
preprint en

Abstract

Modern training treats the transformer residual stream as an anonymous accumulator: any kernel may execute h += \delta, and no record of the write survives the step. Training observes the consequences of internal writes — loss curves, evaluations, activation probes — never the writes themselves. We introduce hytorch, a training loop in which the residual stream is a typed, transactional channel: blocks propose, a pinned policy admits, and every candidate write leaves exactly one 16-byte record — commit, overflow (lost a slot contention) or abort (rule rejection) — so that what the model attempted to write, and why it was refused, is data. Every optimizer step is gated by a durable receipt from an embedded transactional store (Hyphae); a CPU-only verifier replays audited microbatches bit-for-bit against a pinned software reference; the same reference binary admits NVIDIA (sm_89/sm_90), AMD (gfx950) and, through a two-phase policy, AWS Trainium2 kernels on 200,000 adversarial cases with bit-identical verdicts and residuals. Fault injection over real training spills is detected 36/36 with the correct class. The instrument produced two results. Our preregistered protocol reversed our own headline: a single-LR -44.9% perplexity win, replicated across vendors, became a 2.9× loss to the properly tuned dense twin under the preregistered LR sweep, and we publish the reversal. And at nanochat-d20 scale the ledger recorded a silent channel collapse: commit rate 0.066%→0 by step 200, the write codebook frozen bit-exactly, while validation loss improved on schedule through the architecture's declared bypass paths. Neither loss, evaluation nor activation probes distinguish an expensive pathway from a dead one; the write records do, mechanically, during training. We are explicit about what this is and is not. hytorch is an instrumentation and verification framework, not an efficient training architecture: the channel configuration we tested (at most 160 scalar degrees of freedom per token per layer) does not carry a language model — the cataloged d20 model is at chance on knowledge tasks — and costs 12–29× the twin's wall clock; its headline capacity number comes from a run whose STE gradient we later found to be mis-scaled, and the corrected rerun does not yet exist. What the evidence supports is narrower and, we argue, more useful: a training process whose state transitions through the typed channel are witnessed, receipted and replayable. Version 1.1 (5 September 2026) supersedes v1.0 (4 September 2026) after four external reviews; the review synthesis and the list of changes are in the repository (paper/notes/REVIEWS-2026-09.md). Code and evidence: https://github.com/Hyphae-Research-Foundation/hytorch (release v1.1). Source code is licensed Apache-2.0; this paper, its figures and the results prose CC BY-SA 4.0. Every number in the paper traces to a file under results/ or a record key in a Hyphae ledger.

Zenodo (CERN European Organization for Nuclear Research)
Hyams Foundation (US)
Peace, Justice and strong institutions
Security and Verification in Computing
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.