When Mamba Needs Attention
We ask whether a linear-time state-space (Mamba) encoder can drive routing decisions as well as a quadratic-time attention encoder on Euclidean TSP-100, under identical POMO reinforcement learning and a byte-identical decoder, with only the encoder swapped and parameter counts matched. The obvious recipe—a Hilbert-curve serialization scanned by a unidirectional Mamba—is killed: at a matched 250k-step budget it reaches a 7.50 ± 1.03 single-trajectory optimality gap over three seeds versus attention's 2.95%. Diagnostics localize the failure to a short, one-directional receptive field; an ordering ablation shows a locality prior confers no benefit for causal scans and roughly doubles seed variance. Repair follows the diagnosis: pooled global channels fail, a param-matched bidirectional scan reaches multistart parity with attention (1.94 ± 0.09 vs 1.91) and cuts seed variance tenfold, and a hybrid inserting one attention layer into a bidirectional-Mamba stack (4.5% fewer parameters) beats attention on multistart tour quality at a converged 500k budget (worst hybrid seed 1.365 < best attention 1.589). All claims are scoped to N=100; the O(N) payoff remains untested. Code, curves, and pre-registered decision rules: github.com/Umaraslam66/serpentine
Authors
- Umar Aslam
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-07-23
- DOI
- https://doi.org/10.5281/zenodo.21517282
- Primary Topic
- Reinforcement Learning in Robotics
- Type
- preprint