High Capacity and Dynamic Accessibility in Associative Memory Networks with Context-Dependent Neuronal and Synaptic Gating

Biological memory is known to be flexible—memory formation and recall depend on factors such as the behavioral context of the organism. However, this property is often ignored in associative memory models, leaving it unclear how memories can be organized and recalled when subject to contextual control. Because of the lack of a rigorous analytical framework, it is also unknown how contextual control affects memory stability, storage capacity, and information content. Here, we bring the dynamic nature of memory to the fore by introducing a novel model of associative memory, which we refer to as the context-modular memory network. In our model, stored memory patterns are associated to one of several background network states, or contexts. Memories are accessible when their corresponding context is active, and are otherwise inaccessible. Context modulates the effective network connectivity by imposing a specific configuration of neuronal and synaptic gating—gated neurons (synapses) have their activity (weights) momentarily silenced, thereby reducing interference from memories belonging to other contexts. Memory patterns are randomly and independently chosen, while neuronal and synaptic gates may be selected randomly or optimized through a process of contextual synaptic refinement. Through analytic and numerical results, we show that context-modular memory networks can exhibit both improved memory capacity and differential control of memory stability with random gating (especially for neuronal gating). For contextual synaptic refinement, we devise a method in which synapses are gated off for a given context if they destabilize the memory patterns in that context, drastically improving memory capacity and enabling even more precise control over memory stability. Notably, synaptic refinement allows for patterns to be accessible in multiple contexts, stabilizing memory patterns even for weight matrices that alone do not contain any information about the memory patterns, such as Gaussian random matrices. Overall, our model integrates recent ideas about context-dependent memory organization with classic associative memory models and proposes a rigorous theory which can act as a framework for future work. Furthermore, our work carries important implications for the understanding of biological memory storage and recall in the brain, such as highlighting an intriguing trade-off between memory capacity and accessibility.