On the SAGA Algorithm with Decreasing Step
Abstract. Stochastic optimization naturally appears in many application areas, including machine learning. Our goal is to go further in the analysis of the stochastic average gradient accelerated (SAGA) algorithm. To achieve this, we introduce a new [Formula: see text]-SAGA algorithm which interpolates between the stochastic gradient descent ([Formula: see text]) and the SAGA algorithm ([Formula: see text]). First, we investigate the almost sure convergence of this new algorithm with decreasing step which allows us to avoid the restrictive strong convexity and Lipschitz gradient hypotheses associated with the objective function. Second, we establish a central limit theorem for the [Formula: see text]-SAGA algorithm. Finally, we provide the nonasymptotic [Formula: see text] rates of convergence.
Authors
- Bernard Bercu (ORCID: https://orcid.org/0000-0002-8118-5232)
- Eméric Gbaguidi
- Luis Fredes
Institutions
- Université de Bordeaux (FR)
- Institut de Mathématiques de Bordeaux (FR)
Publication Details
- Journal
- SIAM Journal on Control and Optimization
- Published
- 2026-11-16
- DOI
- https://doi.org/10.1137/25m179748x
- Citations
- 1
- Primary Topic
- Stochastic Gradient Optimization Techniques
- Type
- article
- Field-Weighted Citation Impact
- 0.00
Funders
- Agence Nationale de la Recherche