pFedKD: Personalized Federated Learning Meet Knowledge Distillation

Traditional Federated Learning (FL) ensures privacy and security by letting clients jointly learn a model without sharing data with other clients. However, statistical heterogeneity caused each client to have an unsatisfactory personalization ability. To mitigate the problem, we propose pFedKD in this paper, a personalized Federated Learning algorithm with knowledge distillation. During FL training, pFedKD identifies the optimal round with the best local accuracy for each client as the teacher model for knowledge distillation while treating the final round as the student model. Then, the knowledge distillation module extracts desired information for each client's local model, ensuring the accuracy and efficiency of pFedKD. Finally, our extensive experiments on Cifar10 outperform vanilla FedAvg, FedProx, and other SOTA personalized FL algorithms by at least 1.2%, underscoring the superiority of pFedKD.