FedHybrid: Unifying Aggregation Strategies to Optimize Federated Learning on Non-IID Dataset

Federated Learning (FL) allows the execution of Machine Learning at each device while keeping the data at that device. One of the key research difficulties in FL is the development of good aggregation schemes especially when the data is non-IID. This paper introduces a novel aggregation technique, FedHybrid, which combines the strengths of three existing methods:, FedAvg, FedProx, and FedScafold. The proposed method is referred to as FedHybrid since it utilizes model averaging from FedAvg, a proximal term from FedProx to cope with data heterogeneity, and control variates from FedScaffold to minimize update variance. The results of FedHybrid have been tested on the MNIST/CIFAR-10 datasets in a FL environment of 100 clients and over 100 rounds. These experiments show that FedHybrid enhances the overall accuracy and the rate of convergence compared to FedAvg, FedProx, and FedScafold. In particular, the proposed method, obtained 94.12% of accuracy in the MNIST and 93.52% in CIFAR-10 datasets surpassing the other techniques. The improved performance of our proposed model in non-IID data scenarios implies that this work is another valuable addition to the implementable list of federated learning algorithms, promising higher convergence speed and expressly higher accuracy when confronted with commonly realistic non-IID data distributions.