Fully Distributed Federated Learning Using a Zero-Gradient-Sum Algorithm

This paper investigates a distributed method for solving the problems of regression and binary classification in the scope of the Zero-Gradient-Sum (ZGS) theory without local minimization. Instead of a unique model for the whole dataset, the proposed algorithm uses an ensemble of smaller models for the dataset samples, modeled as leaderless coordinated agents over a dynamic network. This algorithm relies only on local information and, in contrast to many algorithms in current literature, agents are not required to share the gradients of the local cost function or raw data, which is a convenient characteristic when privacy is a significant concern. The presented learning algorithm uses a finite-time sliding mode controller to reach the ZGS manifold as the system evolves exponentially toward the global optimum. Numerical simulations are conducted to illustrate the effectiveness of the algorithm.