Divisive Normalization as a Mechanism for Stable and Efficient Recurrent Neural Computation

This dissertation develops a unified theoretical framework for understanding the dynamics, stability, and variability of recurrent neural circuits in both biological and artificial systems. We focus on a family of analytically tractable models known as ORGaNICs (Oscillatory Recurrent Gated Neural Integrated Circuits), which inherently implement the canonical computation of divisive normalization (DN). We investigate the consequences of embedding DN within recurrent neural circuits, with particular emphasis on its impact on circuit stability. First, we prove that high-dimensional ORGaNICs are unconditionally stable and can be mapped to a system of coupled damped harmonic oscillators under constrained recurrent connections. We then characterize the dynamics of ORGaNICs with arbitrary recurrent connections via perturbation theory. We find that the transition to instability is preceded by critical slowing down in the neural dynamics, the onset of which co-occurs with the breakdown of normalization in the neural responses. This stability property enables us to train ORGaNICs on RNN benchmarks, achieving performance competitive with the state of the art. Finally, we develop a large-scale retinotopic model of the primary visual cortex (V1) based on stochastic ORGaNICs. We also derive closed-form solutions for the power spectral density of stochastic linear time-invariant systems driven by Gaussian white noise. Finally, we extend the theory of level crossings of stationary Gaussian processes by deriving closed-form expressions for the variance and Fano factor of arbitrary level up-crossings, previously only known for the mean level. Collectively, this thesis offers novel analytical tools and a cohesive perspective for exploring how neural circuits process information reliably.