Saturation Traps and Learned Conventions in Emergent Analogue Communication
Gradient training can commit a learned communication code, within 200 steps, to a form provably poor for the very objective it minimises. We show this in two jointly trained recurrent networks of 4,000 leaky saturating cells that communicate through a noisy analogue wire. When four steps must carry eight bits, every code with exactly two amplitude values per step has a cross-entropy of at least 4 bits per object; yet training from scratch at noise σ = 0.3 ends at or near this floor in eight of ten seeds (4.02-4.21 bits), while the same networks led through low noise stay multi-level at 2.7-2.9 bits, near the channel limit of 2.54. Trapped codes deliver 69-73% of what the channel permits, multi-level ones 94-97%, and commitment comes at half the noise where two levels become optimal. What governs this saturation trap is the early pre-limiter gain: across a grid of learning rate and initial wire scale, no run is trapped where the median gain at step 100 stays near 2 and every run is trapped above 34; in all 32 trapped runs the gain reaches 2 at or before the code commits; a gain penalty of 0.005 prevents the trap and a rate of 0.001 largely does, unlike surrogate gradients and rescaling. GRU networks do not reach the fully binary floor, but the path dependence extends to them. Discreteness follows the limiter: a peak small relative to the readout's scale makes the code two-level under noise, and without a power cost even without noise; a large peak never does. In a spatial game, a wire swapped in from another episode sends the listener to the landmark that the swapped message names (0.99 of eligible episodes in 13 channels; 11 of 14 in seven new pairs); one channel per set points to a place instead. With each other's messages, independently trained pairs show no established advantage over control recordings (paired difference +0.005, 95% interval -0.05 to 0.09, over 14 pairs). Preprint, version 1. Code and data are available from the author on request: [email protected]. The use of AI agents in this work is described in a footnote on the first page and in the section "Use of AI agents".
Authors
- Vladimir Koshkin
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-11-04
- DOI
- https://doi.org/10.5281/zenodo.23945049
- Primary Topic
- Neural Networks and Applications
- Type
- preprint