A Dual Accelerated Method for Online Stochastic Distributed Averaging: From Consensus to Decentralized Policy Evaluation

Motivated by decentralized sensing and policy evaluation problems, we consider a particular type of distributed stochastic optimization problem over a network, called the online stochastic distributed averaging problem. We design and analyze a dual-based, Polyak–Ruppert averaged method for this problem. We show that the proposed algorithm attains an accelerated deterministic error depending optimally on the condition number of the network, and also that it has optimal stochastic error. This improves on the guarantees of state-of-the-art distributed stochastic optimization algorithms when specialized to this setting, and yields—among other things—corollaries for decentralized policy evaluation. Numerical experiments validate our theoretical results and demonstrate that our approach outperforms existing methods in finite-sample scenarios on several natural network topologies.