On the SAGA Algorithm with Decreasing Step

Abstract. Stochastic optimization naturally appears in many application areas, including machine learning. Our goal is to go further in the analysis of the stochastic average gradient accelerated (SAGA) algorithm. To achieve this, we introduce a new [Formula: see text]-SAGA algorithm which interpolates between the stochastic gradient descent ([Formula: see text]) and the SAGA algorithm ([Formula: see text]). First, we investigate the almost sure convergence of this new algorithm with decreasing step which allows us to avoid the restrictive strong convexity and Lipschitz gradient hypotheses associated with the objective function. Second, we establish a central limit theorem for the [Formula: see text]-SAGA algorithm. Finally, we provide the nonasymptotic [Formula: see text] rates of convergence.

Authors

Institutions

Publication Details

Journal
SIAM Journal on Control and Optimization
Published
2026-11-16
DOI
https://doi.org/10.1137/25m179748x
Citations
1
Primary Topic
Stochastic Gradient Optimization Techniques
Type
article
Field-Weighted Citation Impact
0.00

Funders

Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

On the SAGA Algorithm with Decreasing Step

Bernard Bercu, Eméric Gbaguidi, Luis Fredes
1 citations
SIAM Journal on Control and Optimization
Stochastic Gradient Optimization Techniques
article

On the SAGA Algorithm with Decreasing Step

Bernard Bercu, Eméric Gbaguidi, Luis Fredes
article en
1 citations

Abstract

Abstract. Stochastic optimization naturally appears in many application areas, including machine learning. Our goal is to go further in the analysis of the stochastic average gradient accelerated (SAGA) algorithm. To achieve this, we introduce a new [Formula: see text]-SAGA algorithm which interpolates between the stochastic gradient descent ([Formula: see text]) and the SAGA algorithm ([Formula: see text]). First, we investigate the almost sure convergence of this new algorithm with decreasing step which allows us to avoid the restrictive strong convexity and Lipschitz gradient hypotheses associated with the objective function. Second, we establish a central limit theorem for the [Formula: see text]-SAGA algorithm. Finally, we provide the nonasymptotic [Formula: see text] rates of convergence.

SIAM Journal on Control and OptimizationVol. 64(6)
Université de Bordeaux (FR), Institut de Mathématiques de Bordeaux (FR)
Agence Nationale de la Recherche
Stochastic Gradient Optimization Techniques
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon