Slow Passage Through a Saddle-Node Bifurcation: A Unified Framework for Stochastic Thermodynamics and Learning Generalization Singularities

We propose a unied framework that links two recent results from disjoint elds: the v 2/3scaling of the excess work in nite-time rst-order phase transitions of the kinetic CurieWeissmodel [WCQ25], and the identication of the interpolation threshold in overparameterized lin-ear regression as a non-equilibrium phase transition with broken ergodicity [LG25]. We showthat both phenomena are instances of slow passage through a saddlenode bifurcation in ahigh-dimensional stochastic system whose deterministic mean-eld reduction admits a one-dimensional unstable manifold. We render this connection mathematically precise by deriv-ing a canonical normal form du/ds = u2 + s in both settings via center-manifold reductionwith adiabatic elimination of fast modes, recovering the Airy-function representation that con-trols the v−1/3 delay-time scaling and the v2/3 swept-area scaling. We then prove that theuctuationdissipation theorem (FDT) breakdown observed in the overparameterized regimeof dynamical mean-eld theory (DMFT) for stochastic gradient descent on linear regression isalgebraically equivalent to the persistence of a 1/(iω) pole in the linear response of a Langevinsystem possessing a zero-eigenvalue mode of the eective Hessian, with residue equal to thefractional codimension of the row space of the design matrix. The DMFT analysis extendsfrom linear regression to deep neural networks in two regimes: the lazy / neural-tangent-kernelregime, where the framework applies layer-by-layer with a parameter-weighted layer-additive f-sum rule, and the feature-learning regime, where the order parameter becomes time-dependentand the lazy-to-rich transition is itself an instance of the Airy normal form.The framework admits a natural categorication that we develop in parallel: the orderparameter ηerg = (α − 1)/α is lifted to a functor valued in K-theory classes, the f-sum rulebecomes an index pairing on a split-exact sequence in a response category Resp(α), the (N, v)phase diagram is realized as a Conley persistence module, and the Airy normal form is identiedas the universal 1-morphism through a saddlenode object in a 2-category GSP of generalizedslow passages. The numerical content of the original theory is recovered as the decategorication.Four concrete predictions follow from this analytical and categorical structure: (i) thecrossover line in the (N, v) plane of the kinetic CurieWeiss model is logarithmic in v withslope determined by the cusp-catastrophe barrier exponent; (ii) the dynamical critical exponents(σ, ϕ) = (1, 2) asserted for ridgeless linear regression are consistent with the rst-order pole ofthe HastieMontanariRossetTibshirani static formula [Has+22] when σ is correctly read asthe pole order and ϕ as the dynamical-time exponent, with Egen(t, α = 1) ∝√t at criticalityand an explicit closed-form scaling function; (iii) grokking in algorithmic neural networks shouldexhibit Airy-type delay statistics with τgrok ∼ v−1/3eff , sharpened by the categorication to the1full Airy distribution rather than the mean alone; and (iv) a secondary K-invariant computedfrom the categoried Chern character should distinguish overparameterized linear regression,grokking, and anisotropic-covariance regimes even when these share the same ηerg.

Authors

Institutions

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-05-23
DOI
https://doi.org/10.5281/zenodo.20350522
Primary Topic
Stochastic Gradient Optimization Techniques
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
preprint

Slow Passage Through a Saddle-Node Bifurcation: A Unified Framework for Stochastic Thermodynamics and Learning Generalization Singularities

Alfredo Sepulveda-Jimenez
Zenodo (CERN European Organization for Nuclear Research)
Stochastic Gradient Optimization Techniques
preprint

Slow Passage Through a Saddle-Node Bifurcation: A Unified Framework for Stochastic Thermodynamics and Learning Generalization Singularities

Alfredo Sepulveda-Jimenez
preprint en

Abstract

We propose a unied framework that links two recent results from disjoint elds: the v 2/3scaling of the excess work in nite-time rst-order phase transitions of the kinetic CurieWeissmodel [WCQ25], and the identication of the interpolation threshold in overparameterized lin-ear regression as a non-equilibrium phase transition with broken ergodicity [LG25]. We showthat both phenomena are instances of slow passage through a saddlenode bifurcation in ahigh-dimensional stochastic system whose deterministic mean-eld reduction admits a one-dimensional unstable manifold. We render this connection mathematically precise by deriv-ing a canonical normal form du/ds = u2 + s in both settings via center-manifold reductionwith adiabatic elimination of fast modes, recovering the Airy-function representation that con-trols the v−1/3 delay-time scaling and the v2/3 swept-area scaling. We then prove that theuctuationdissipation theorem (FDT) breakdown observed in the overparameterized regimeof dynamical mean-eld theory (DMFT) for stochastic gradient descent on linear regression isalgebraically equivalent to the persistence of a 1/(iω) pole in the linear response of a Langevinsystem possessing a zero-eigenvalue mode of the eective Hessian, with residue equal to thefractional codimension of the row space of the design matrix. The DMFT analysis extendsfrom linear regression to deep neural networks in two regimes: the lazy / neural-tangent-kernelregime, where the framework applies layer-by-layer with a parameter-weighted layer-additive f-sum rule, and the feature-learning regime, where the order parameter becomes time-dependentand the lazy-to-rich transition is itself an instance of the Airy normal form.The framework admits a natural categorication that we develop in parallel: the orderparameter ηerg = (α − 1)/α is lifted to a functor valued in K-theory classes, the f-sum rulebecomes an index pairing on a split-exact sequence in a response category Resp(α), the (N, v)phase diagram is realized as a Conley persistence module, and the Airy normal form is identiedas the universal 1-morphism through a saddlenode object in a 2-category GSP of generalizedslow passages. The numerical content of the original theory is recovered as the decategorication.Four concrete predictions follow from this analytical and categorical structure: (i) thecrossover line in the (N, v) plane of the kinetic CurieWeiss model is logarithmic in v withslope determined by the cusp-catastrophe barrier exponent; (ii) the dynamical critical exponents(σ, ϕ) = (1, 2) asserted for ridgeless linear regression are consistent with the rst-order pole ofthe HastieMontanariRossetTibshirani static formula [Has+22] when σ is correctly read asthe pole order and ϕ as the dynamical-time exponent, with Egen(t, α = 1) ∝√t at criticalityand an explicit closed-form scaling function; (iii) grokking in algorithmic neural networks shouldexhibit Airy-type delay statistics with τgrok ∼ v−1/3eff , sharpened by the categorication to the1full Airy distribution rather than the mean alone; and (iv) a secondary K-invariant computedfrom the categoried Chern character should distinguish overparameterized linear regression,grokking, and anisotropic-covariance regimes even when these share the same ηerg.

Zenodo (CERN European Organization for Nuclear Research)
QED Labs (US)
Stochastic Gradient Optimization Techniques
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.