FedPPO: Reinforcement Learning-Based Client Selection for Federated Learning With Heterogeneous Data

Federated Learning (FL) enables multiple data owners to jointly train a machine learning model, which can improve joint environmental cognitive capability without disclosing their private local data. In FL systems, local clients are often heterogeneous, characterized by different data distribution forms and varying levels of noise patterns. This poses challenges to the convergence and test accuracy of the learned global model. Existing client selection methods failed to effectively address the data heterogeneity problem while considering the impacts of correlations between clients. In this work, we propose an efficient and adaptive client selection scheme with the Federated Proximal Policy Optimization (FedPPO). It utilizes a noisy client filtering method that leverages the principal component analysis and hierarchical clustering to reduce the action search space and accelerate the convergence of Deep Reinforcement Learning (DRL). It then evaluates each client’s contribution by jointly using the accuracy of global model and local models and take the contribution score as the reward. Finally, it leverages a DRL algorithm, PPO, to adaptively select clients with high contributions to participate in training. Theoretical analysis shows that it is guaranteed to converge in an efficient manner. Experimental results on four real-world datasets show that FedPPO significantly outperforms existing approaches of constructing global models with faster convergence speed and higher test accuracy.