FedMKD: Personalized Federated Learning with Memory Knowledge Distillation

In recent years, privacy concerns have been receiving increasing attention. Federated learning (FL) has emerged to address privacy challenges in machine learning. In this framework, a group of clients collaborates with a server, where clients upload model parameters instead of raw data. In FL, a key challenge is the presence of non-independent and identically distributed data among clients. To address this issue, we propose a personalized federated learning algorithm (FedMKD) that fully considers the characteristics of the previous round's model. Our algorithm leverages knowledge distillation and gradient descent to optimize the proportion of global model parameters used during the initialization process. FedMKD combines the strengths of the global and memory models to initialize the local model. It effectively retains historical information during training, enhancing the local model's performance. Extensive experimental results demonstrate the effectiveness of our proposed algorithm in addressing statistical heterogeneity issues. FedMKD achieves up to a 10.66% improvement in test accuracy compared to eight state-of-the-art baselines.