Dynamic Personalized Federated Learning via Representation-Driven Clustering

Clustering Federated Learning (CFL) promotes knowledge sharing by adaptively grouping similar clients while solving the Non-IID problem to provide personalized models with high generalization. However, there are several problems needed to be considered before the practical deployment of CFL: 1) Practical clients often have limited communication capabilities. 2) Dynamically evolving data distributions of clients may leads to unreasonable clustering. 3) Asynchronization will cause clients belonging to the same cluster to fall apart. The above three problems will reduce the convergence speed and damage the accuracy of the model. In this work, we propose a Dynamic Representation-driven Clustering Federated Learning framework (DReCFL) to solve the above three problems. Specifically, DReCFL replaces complex model parameters with data representation for communicating to reduce the communication pressure. In order to gain a reasonable clustering, DReCFL utilizes a Dynamic Client Fuzzy Clustering (DCFC) algorithm, we proposed, to adapt to evolving data distributions. Finally, DReCFL leverages an Adaptive Clustering Threshold (ACT) mechanism, we designed, to craft clustering thresholds based on training progress, ensuring the reliability of client clustering results under asynchrony. Extensive experimental results demonstrate that DReCFL efficiently adapts to the addition of new clients and changes in data distribution, while reducing communication overhead by 22.70%-83.33% compared to state-of-the-art (SOTA) personalized federated learning methods.