Combining Centralized and Decentralized Federated Learning: An Integrated Framework

Centralized federated learning (CFL) can lead to communication bottleneck because of bottlenecks, and decentralized federated learning (DFL) may cause the training trapped in the local optimum due to the lack of global views. In this paper, we propose an integrated framework of federated learning that combines the advantages of CFL and DFL. First, we design an algorithm by periodically performing global aggregation, neighbor consensus and local stochastic gradient descent (SGD). Second, we prove the convergence of the designed algorithm and analyze the effect of two stages (global aggregation and neighbor consensus) on the convergence. Finally, experiment results based on the CIFAR-10 dataset show that the integrated federated learning (IFL) accelerates the convergence and has higher accuracy compared to CFL and DFL.