Complexity and performance for two classes of noise-tolerant first-order algorithms
Two classes of algorithms for optimization in the presence of noise are presented, that<br/>do not require the evaluation of the objective function. The first generalizes the well-<br/>known Adagrad method. Its complexity is then analyzed as a function of its parameters.<br/>A second class of algorithms is then derived whose complexity<br/>is at least as good as that of the first class. Initial numerical experiments on finite-sum<br/>problems arising from deep-learning applications suggest that methods of the second<br/>class may outperform those of the first.
Authors
- Philippe L. Toint (ORCID: https://orcid.org/0000-0002-6166-1860)
- S. Gratton
- Sadok Jerad
Institutions
- Université Toulouse III - Paul Sabatier (FR)
- University of Namur (BE)
- Université Fédérale de Toulouse Midi-Pyrénées (FR)
- Université Toulouse-I-Capitole (FR)
- Institut de Recherche en Informatique de Toulouse (FR)
- Université Toulouse - Jean Jaurès (FR)
- Institut Polytechnique de Bordeaux (FR)
Publication Details
- Journal
- Optimization methods & software
- Published
- 2025-09-29
- DOI
- https://doi.org/10.1080/10556788.2025.2532736
- Primary Topic
- Stochastic Gradient Optimization Techniques
- Type
- article
- Field-Weighted Citation Impact
- 0.00
Funders
- Agence Nationale de la Recherche
- Centre National de la Recherche Scientifique
- European Regional Development Fund