The Infinite Volume Limit of the Hopfield Neural Networks in the Replica-Symmetric Regime

Abstract Despite extensive research over the past five decades, a rigorous determination of the infinite volume limit of the free energy of the Hopfield model with an extensive number of patterns has remained an outstanding open problem as standard methods from spin glass theory have so far proved inadequate for this purpose. While a complete proof for finite, nonvanishing, loads is still out of reach, here we develop a novel interpolation strategy that allows us to express the free energy of the Hopfield model in terms of a linear combination of the free energies of hard and soft spin glass models along with a remainder contribution. The payoff of this interpolation scheme is twofold: (i) As the remainder involves nonnegative fluctuations of the order parameters (and it vanishes under a concentration assumption on their means where we consistently recover the replica-symmetric picture) such an expression finally constitutes a rigorous bound on the exact free energy, rather than just an approximation, as it was the case for earlier interpolation procedures á la Guerra.(ii) As for the spin glass models the existence of the infinite volume free energy has been previously determined by the Guerra–Toninelli scheme, this novel interpolation allows us to extend its validity to the Hopfield model too. Yet, as the equivalence between the Hopfield model and mixtures of spin glasses is currently well established under replica symmetry and up to the first step of replica symmetry breaking, at present our approach can be rigorously implemented only within these regimes. In this work, by inspecting how the order parameters have to self-average around their means, we restrict our attention to the replica-symmetric regime.