Decentralized Nonconvex Robust Optimization Over Unsafe Multiagent Systems: System Modeling, Utility, Resilience, and Privacy Analysis

Privacy leakage and Byzantine issues are two adverse factors to optimization and learning processes of multiagent systems (MASs). Considering an unsafe MAS with these two issues, this article targets the resolution of a category of nonconvex optimization problems under the Polyak-Łojasiewicz (P-Ł) condition. To address this problem, we first identify and construct the unsafe MAS model. Under this kind of unfavorable MASs, we mask the local gradients with Gaussian noise and adopt a resilient aggregation method, self-centered clipping (SCC), to design a differentially private (DP) and Byzantine-resilient (BR) decentralized stochastic gradient algorithm, dubbed DP-SCC-PL, aiming to address a class of nonconvex optimization problems in the presence of both privacy leakage and Byzantine issues. The convergence analysis of DP-SCC-PL is challenging, as the convergence error arises from the coupled effects of DP and BR mechanisms, as well as the nonconvex relaxation, which is resolved via seeking the contraction relationships among the disagreement measure of reliable agents before and after the SCC aggregation, together with the optimal gap. Theoretical results not only reveal the trilemma between algorithm utility, resilience, and privacy, but also show that DP-SCC-PL can achieve consensus among all reliable agents. It has also been proven that if there are no privacy issues and Byzantine agents, then the asymptotic exact convergence can be recovered. Numerical experiments verify the utility, resilience, and privacy of DP-SCC-PL by tackling a nonconvex optimization problem satisfying the P-Ł condition under various Byzantine attacks.