Attention Conservation in Transformers: Testing for Global Flow Invariants Across Softmax, Residual, and MLP Pathways

This paper investigates whether softmax normalization in Transformer attention corresponds to a global conservation law, or only a local probabilistic constraint. Self-attention guarantees that outgoing attention weights from each query sum to one, inviting comparison with conservation constraints in flow networks and electrical circuits. We distinguish local normalization from cross-layer conservation and formulate an empirically testable protocol across three progressively complete Transformer representations: attention-only propagation, attention with residual connections, and the full attention-residual-MLP block. We hypothesize that softmax normalization alone does not imply global conservation, because the residual stream accumulates additive contributions rather than redistributing a fixed quantity. Candidate flow quantities are defined (attention mass, activation-weighted flow, feature-specific flow), alongside a conservation profile metric and an intervention protocol (token ablation, head ablation, activation patching, residual perturbation). A worked numerical case study illustrates the protocol's sensitivity across the three architectural systems. The question is situated against attention rollout and attention flow (Abnar and Zuidema, 2020), Flowformer's explicitly imposed flow conservation (Wu et al., 2022), and the descriptive, non-conservation-claiming use of circuit language in mechanistic interpretability (Olah et al., 2020; Elhage et al., 2021). Conservation is treated throughout as an open empirical hypothesis rather than an assumed structural fact.