Dynamic Clustering for Federated Learning: A Reinforcement Learning Approach to Minimize Time Divergence

Federated learning (FL) offers a decentralized approach to training machine learning models, allowing data to remain on local devices and preserving privacy. However, the decentralized nature of $\mathbf{F L}$ introduces challenges such as heterogeneous device performance, leading to straggler devices that slow down the training process. Traditional clustering methods, whether static or dynamic, often fail to adapt effectively to these variations. To address this, the paper proposes FedDy, an advanced algorithm incorporating reinforcement learning (RL) and hybrid clustering techniques. FedDy begins with k-means clustering and dynamically selects the optimal clustering method in each round, guided by RL. Silhouette analysis helps determine the optimal number of clusters, while a hybrid metric considers upload times, computation times, and model divergences to ensure balanced clusters and efficient training. The experiments on the Fashion MNIST dataset demonstrate that FedDy reduces training time by $69.1 \%$ compared to FedAvg, with a similar accuracy of 90.27${\$}$. The enhanced version, FedDy_Enh, further reduces training time by 65.8% compared to FedAvg and 32.3% compared to FedStat, achieving the highest accuracy of 90.74%.