Fixed point geometry in chaotic neural networks

Understanding the high-dimensional chaotic dynamics occurring in complex biological systems such as recurrent neural networks or ecosystems remains a conceptual challenge. For low-dimensional dynamics, fixed points provide the geometric scaffold of the dynamics. However, in high-dimensional systems, even the location of fixed points is unknown. Here, we analytically determine the number and distribution of fixed points for a canonical model of a recurrent neural network that exhibits high-dimensional chaos. This distribution reveals that fixed points and dynamics are confined to separate shells in state space. Furthermore, the distribution enables us to determine the eigenvalue spectra of the Jacobian at the fixed points, showing that each fixed point has a low-dimensional unstable manifold. Despite the radial separation of fixed points and dynamics, we find that the principal components of fixed points and dynamics align and that nearby fixed points act as partially attracting landmarks for the dynamics. Our results provide a detailed characterization of the fixed point geometry and its interplay with the dynamics, thereby paving the way towards a geometric understanding of high-dimensional chaos through their skeleton of unstable fixed points.