Adam Optimizer at the Edge of Stability: Exact Thresholds for Period-Two Oscillations

On f(w) = (λ/2)w², Adam without bias correction has an exact period-two orbit wt = (−1)tA with A = η(1−β₁)/(2(1+β₁)) − ε/λ. On it the second moment equals gt², and the preconditioned curvature equals the adaptive edge-of-stability threshold 2(1+β₁)/(η(1−β₁)) of Cohen et al. Bock and Weiß (2019) found this orbit at ε = 0. They observed that its multipliers depend on (β₁, β₂) only, and their plots show it unstable for β₁, β₂ near 1. They obtained lengthy symbolic eigenvalue expressions but left an analytic description of the stability region unresolved. Fong and Yang (2026) give the orbit for ε > 0. We prove that the Floquet multipliers are the squares of the roots of an explicit cubic in (β₁, β₂, θ), where θ = A/A⋆ ∈ (0, 1] measures ε. Apart from the degenerate sign-descent corner, the orbit is linearly asymptotically stable if and only if (1−β₁)(1−β₁β₂) > 2θβ₁(1+β₂), with an explicit unit-circle factorization at the boundary. At ε = 0 it is stable for every β₂ ∈ [0, 1) when 0 < β₁ ≤ 3 − 2√2 ≈ 0.172, and strictly linearly unstable for every β₂ when β₁ > 1/3. At (0.9, 0.999) the dominant linear growth factor is 1.0494 per step. At these defaults, a positive ε stabilises the orbit only when λA⋆ exceeds ε by less than 0.28%. The same cubic, with a gradient elasticity κ in place of 1, governs the symmetric orbit of every even loss. A unified, division-free criterion gives its full stability interval, including both real and oscillatory boundaries. At ε = 0 this reduces to 0 < κ < κ⋆ outside the degenerate corner; κ⋆ = 0.9906 at the defaults, so the quadratic lies just on the unstable side. On quadratics in ℝd, every sign cycle of coordinate-wise Adam is unstable at the defaults, and these cycles persist, still unstable, near every non-degenerate minimum of a C3 loss for small step size. A decoupled weight decay stabilises the orbit at the defaults only if it removes more than 97% of the weight per step. At ε = 0 the loss of stability is a sub-critical Neimark–Sacker bifurcation for every β₂ ≠ 2 − √3: we give the first Lyapunov coefficient in closed form and prove it positive. For ε > 0 the criticality can change; along β₁ = 0.9 it does so exactly once, at β₂ = 0.98948. In one dimension the two-cycle and its Floquet cubic are exact for losses without symmetry. Simulations at the defaults exhibit sustained irregular motion, with a finite-time largest Lyapunov exponent of about 5 × 10−3 per step. On these trajectories the preconditioned curvature averages 0.93 of the threshold and exceeds it on 18.5% of the steps, and the loss averages 1.17 times the loss on the orbit. Below the stability boundary, distinct initializations provide numerical evidence of coexistence with a large-amplitude regime. Dedication. The mathematical analysis developed in this work is dedicated with unconditional love and gratitude to Giovanna Elizabeth Pereira de Matos Costa, the author’s mother. Throughout my entire life, on the bench and at home, you taught by example that true judgment never rests on appearances or conventional defaults, but looks past the surface to uncover hidden flaws with rigor, fairness, and unwavering integrity. Just as this paper exposes the instability beneath an assumed equilibrium, may these theorems stand as permanent tribute to the judge who taught me to seek the exact truth—and to the mother whose love remains the one true constant in my life.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-11-15
DOI
https://doi.org/10.5281/zenodo.24239329
Primary Topic
Stochastic Gradient Optimization Techniques
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Adam Optimizer at the Edge of Stability: Exact Thresholds for Period-Two Oscillations

Gabriel Paiva
Zenodo (CERN European Organization for Nuclear Research)
Stochastic Gradient Optimization Techniques
preprint

Adam Optimizer at the Edge of Stability: Exact Thresholds for Period-Two Oscillations

Gabriel Paiva
preprint en

Abstract

On f(w) = (λ/2)w², Adam without bias correction has an exact period-two orbit wt = (−1)tA with A = η(1−β₁)/(2(1+β₁)) − ε/λ. On it the second moment equals gt², and the preconditioned curvature equals the adaptive edge-of-stability threshold 2(1+β₁)/(η(1−β₁)) of Cohen et al. Bock and Weiß (2019) found this orbit at ε = 0. They observed that its multipliers depend on (β₁, β₂) only, and their plots show it unstable for β₁, β₂ near 1. They obtained lengthy symbolic eigenvalue expressions but left an analytic description of the stability region unresolved. Fong and Yang (2026) give the orbit for ε > 0. We prove that the Floquet multipliers are the squares of the roots of an explicit cubic in (β₁, β₂, θ), where θ = A/A⋆ ∈ (0, 1] measures ε. Apart from the degenerate sign-descent corner, the orbit is linearly asymptotically stable if and only if (1−β₁)(1−β₁β₂) > 2θβ₁(1+β₂), with an explicit unit-circle factorization at the boundary. At ε = 0 it is stable for every β₂ ∈ [0, 1) when 0 < β₁ ≤ 3 − 2√2 ≈ 0.172, and strictly linearly unstable for every β₂ when β₁ > 1/3. At (0.9, 0.999) the dominant linear growth factor is 1.0494 per step. At these defaults, a positive ε stabilises the orbit only when λA⋆ exceeds ε by less than 0.28%. The same cubic, with a gradient elasticity κ in place of 1, governs the symmetric orbit of every even loss. A unified, division-free criterion gives its full stability interval, including both real and oscillatory boundaries. At ε = 0 this reduces to 0 < κ < κ⋆ outside the degenerate corner; κ⋆ = 0.9906 at the defaults, so the quadratic lies just on the unstable side. On quadratics in ℝd, every sign cycle of coordinate-wise Adam is unstable at the defaults, and these cycles persist, still unstable, near every non-degenerate minimum of a C3 loss for small step size. A decoupled weight decay stabilises the orbit at the defaults only if it removes more than 97% of the weight per step. At ε = 0 the loss of stability is a sub-critical Neimark–Sacker bifurcation for every β₂ ≠ 2 − √3: we give the first Lyapunov coefficient in closed form and prove it positive. For ε > 0 the criticality can change; along β₁ = 0.9 it does so exactly once, at β₂ = 0.98948. In one dimension the two-cycle and its Floquet cubic are exact for losses without symmetry. Simulations at the defaults exhibit sustained irregular motion, with a finite-time largest Lyapunov exponent of about 5 × 10−3 per step. On these trajectories the preconditioned curvature averages 0.93 of the threshold and exceeds it on 18.5% of the steps, and the loss averages 1.17 times the loss on the orbit. Below the stability boundary, distinct initializations provide numerical evidence of coexistence with a large-amplitude regime. Dedication. The mathematical analysis developed in this work is dedicated with unconditional love and gratitude to Giovanna Elizabeth Pereira de Matos Costa, the author’s mother. Throughout my entire life, on the bench and at home, you taught by example that true judgment never rests on appearances or conventional defaults, but looks past the surface to uncover hidden flaws with rigor, fairness, and unwavering integrity. Just as this paper exposes the instability beneath an assumed equilibrium, may these theorems stand as permanent tribute to the judge who taught me to seek the exact truth—and to the mother whose love remains the one true constant in my life.

Zenodo (CERN European Organization for Nuclear Research)
Stochastic Gradient Optimization Techniques
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon