A Relocated Dependency Is Not a Vanished One: What the Neural-Cellular-Automaton Result on Integer Multiplication Shows About Long-Range Dependency, Why the Neural GPU Line Already Bounds It, and the Controls That Would Separate Locality, Supervision
(c) 2026 Pranay Mahendrakar. Licensed under CC BY 4.0. Life of Research Multi-digit multiplication is a standing failure case for neural sequence models, and a 2025 reverse-engineering study traced the failure of standard fine-tuning to long-range dependencies that the model never learns. A 2026 paper rejects that diagnosis: it lays two binary integers out as an outer-product grid, trains a 321-parameter neural cellular automaton on single update steps, and reports exact products up to 4,096 bits after training on 2 to 6 bits, which it reads as proof that the long-range dependency was a mirage of the chosen representation and architecture. In this analysis the author reviews and synthesises published results to show that the two papers are mostly not in conflict, because "long-range dependency" names three different things. The functional dependency of a high product bit on many input bits is a property of multiplication under any encoding; the reach of a single update step is a property of representation and architecture, and the grid removes it; the number of sequential steps is the price of that removal, and the automaton pays it, taking about n steps for n-bit operands. The paper then argues that the published record already bounds the mirage claim. Local, iterated networks learned binary multiplication from input-output pairs alone in 2015, including a one-dimensional variant that reached 2,000 bits from 20; the same line failed on long carry chains that uniformly random test inputs almost never contain; and on both sides of the dispute, most of the reliable successes supervise intermediate states. The paper consolidates the measured values in Table 1 and Figure 1, states an attribution procedure as Algorithm 1, and specifies six controls, including a carry-adversarial test set and an end-to-end-trained automaton, that would separate locality, supervision and carry coverage.
Authors
- Pranay M. Mahendrakar (ORCID: https://orcid.org/0009-0003-7224-029X)
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-11-15
- DOI
- https://doi.org/10.5281/zenodo.24235209
- Primary Topic
- Cellular Automata and Applications
- Type
- preprint