Decentralized Online Dual Averaging Optimization Ensuring Differential Privacy and Expected Regret

The decentralized storage feature of big data may potentially expose risks of privacy breaches during real-time information interactions. To address this concern, this study focuses on constrained online learning problems with privacy awareness over generally directed communication networks. Specifically, we develop a novel secure decentralized online algorithm that utilizes the idea of the dual-averaging method as a core dynamic procedure. Different from existing dual-averaging works, the proposed algorithm admits arbitrary initial conditions, which is more flexible and generalized. The algorithm performs a gradient scaling operation on the dual dynamics, which effectively eliminates errors caused by unbalanced communications. The results show that the proposed algorithm ensures both differential privacy and expected regret. Simulations further confirm the correctness of our work.