Beyond Additive Attention: Multiplicative Structural Gating for Sequence Memory

—Standard Recurrent Neural Networks (RNNs) compress historical context into opaque hidden state vectors, while Transformers rely on additive positional biases and O(N2) attention. In this paper, we introduce a lightweight memory architecture that stores token in stances explicitly and retrieves them via Direct-Jump routing. Instead of standard additive attention, we utilize Multiplicative Structural Gating, where inverse hop distance acts as a strict multiplicative prior on memory retrieval.Through ablation studies on a sequence memorization task,we demonstrate that this combination successfully learns sequence representations with 100% accuracy, whereas standard additive attention fails. Furthermore, we in vestigate compressing this instance memory into a fixed V ×V token-type graph. We show that this compression fails because a fixed token-type matrix cannot preserve the sequence order of repeated token instances, and that augmenting it with a predictive surprise gate causes catastrophic runaway feedback loops.