Constant step-size stochastic approximation with delayed updates
In this paper, we consider constant step-size stochastic approximation with delayed updates. For the non-delayed case, it is well known that under appropriate conditions, the discrete-time iterates of stochastic approximation track the trajectory of a continuous-time ordinary differential equation (ODE). For the delayed case, we show in this paper that, under appropriate conditions, the discrete-time iterates track the trajectory of a delay-differential equation (DDE) rather than an ODE. Thus, delayed updates lead to a qualitative change in the behavior of constant step-size stochastic approximation. We present multiple examples to illustrate the qualitative affect of delay and show that increasing the delay is generally destabilizing but, for some systems, it can be stabilizing as well.
Authors
- Aditya Mahajan (ORCID: https://orcid.org/0000-0001-8125-1191)
- Silviu‐Iulian Niculescu (ORCID: https://orcid.org/0000-0002-3444-2566)
- M. Vidyasagar (ORCID: https://orcid.org/0000-0003-1057-1942)
Institutions
- HEC Montréal (CA)
- Centre National de la Recherche Scientifique (FR)
- Université Paris-Saclay (FR)
- Laboratoire des signaux et systèmes (FR)
- CentraleSupélec (FR)
- Indian Institute of Technology Goa (IN)
- Group for Research in Decision Analysis (CA)
- Mila - Quebec Artificial Intelligence Institute (CA)
- International Laboratory on Learning Systems
- McGill University (CA)
- Indian Institute of Technology Hyderabad (IN)
Publication Details
- Published
- 2024-12-16
- DOI
- https://doi.org/10.1109/cdc56724.2024.10886824
- Primary Topic
- Stochastic processes and financial applications
- Type
- article
- Field-Weighted Citation Impact
- 0.00
Funders
- Université Paris-Saclay
- Natural Sciences and Engineering Research Council of Canada
- Science and Engineering Research Board