Heavy-ball differential equation achieves $$O(\varepsilon ^{-7/4})$$ convergence for nonconvex functions

First-order optimization methods for nonconvex functions with Lipschitz continuous gradient and Hessian have been extensively studied. State-of-the-art methods for finding an $$\varepsilon $$ -stationary point within $$O(\varepsilon ^{-{7/4}})$$ or $$\tilde{O}(\varepsilon ^{-{7/4}})$$ gradient evaluations are based on Nesterov’s accelerated gradient descent (AGD) or Polyak’s heavy-ball (HB) method. However, these algorithms employ additional mechanisms, such as restart schemes and negative curvature exploitation, which complicate their behavior and make it challenging to apply them to more advanced settings (e.g., stochastic optimization). As a first step in investigating whether a simple algorithm with $$O(\varepsilon ^{-{7/4}})$$ complexity can be constructed without such additional mechanisms, we study the HB differential equation, a continuous-time analogue of the AGD and HB methods. We prove that its dynamics attain an $$\varepsilon $$ -stationary point within $$O(\varepsilon ^{-{7/4}})$$ time.

Authors

Institutions

Publication Details

Journal
Optimization Letters
Published
2026-01-21
DOI
https://doi.org/10.1007/s11590-025-02270-w
Primary Topic
Stochastic Gradient Optimization Techniques
Type
article
Field-Weighted Citation Impact
0.00

Funders

Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Heavy-ball differential equation achieves $$O(\varepsilon ^{-7/4})$$ convergence for nonconvex functions

Naoki Marumo, Kaito Okamura, Akiko Takeda
Optimization Letters
Stochastic Gradient Optimization Techniques
article

Heavy-ball differential equation achieves $$O(\varepsilon ^{-7/4})$$ convergence for nonconvex functions

Naoki Marumo, Kaito Okamura, Akiko Takeda
article en

Abstract

First-order optimization methods for nonconvex functions with Lipschitz continuous gradient and Hessian have been extensively studied. State-of-the-art methods for finding an $$\varepsilon $$ -stationary point within $$O(\varepsilon ^{-{7/4}})$$ or $$\tilde{O}(\varepsilon ^{-{7/4}})$$ gradient evaluations are based on Nesterov’s accelerated gradient descent (AGD) or Polyak’s heavy-ball (HB) method. However, these algorithms employ additional mechanisms, such as restart schemes and negative curvature exploitation, which complicate their behavior and make it challenging to apply them to more advanced settings (e.g., stochastic optimization). As a first step in investigating whether a simple algorithm with $$O(\varepsilon ^{-{7/4}})$$ complexity can be constructed without such additional mechanisms, we study the HB differential equation, a continuous-time analogue of the AGD and HB methods. We prove that its dynamics attain an $$\varepsilon $$ -stationary point within $$O(\varepsilon ^{-{7/4}})$$ time.

Optimization Letters
RIKEN Center for Advanced Intelligence Project (JP), The University of Tokyo (JP)
University of Tokyo, Japan Society for the Promotion of Science, Core Research for Evolutional Science and Technology
Openalex Percentile: Top 83%
Stochastic Gradient Optimization Techniques
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Heavy-ball differential equation achieves $O(\varepsilon ^{-7/4})$ convergence for nonconvex functions — Naoki Marumo, Kaito Okamura, et al. · Optimization Letters (2026) | TGRS Research Map | TGRS