Thompson Sampling Policy for Dynamic Participating Client Scenario in Federated Learning

This article focuses on training a robust global model under the dynamic participating client scenario in federated learning (FL). Unlike the static FL scenario, this scenario emphasizes that new clients participate into modeling process at specific training round, a situation commonly encountered in Internet of Things (IoT) networks. Particularly, new participating clients may introduce irrelevant data (e.g., label noise, outliers, and distinct data distributions), posing the challenge for training a robust global model. To tackle this challenge, we introduce a novel FL framework called Fed-TS. This framework incorporates the Thompson sampling (TS) policy, a reinforcement learning (RL) approach, to enable biased client participation for both relevant and irrelevant clients during modeling process. Specifically, new clients can participate into modeling process based on their successful probability of causing “small path drift" in the global model. In addition, our framework employs an adaptive threshold using the K-means algorithm to assess “small path drift" in the global model. During the experiments, we investigate the impact of hyper-parameters on the performance of biased client participation for both relevant and irrelevant clients. Compared to other schemes or frameworks, our framework ensures a robust global model with high accuracy.