FedKDF: Personalized Federated Learning Based on Knowledge Distillation and Fisher Information Matrix

Federated learning, as a distributed machine learning paradigm, can effectively protect data privacy and reduce data transmission costs, which makes it widely applicable in fields that require high data privacy. However, the inconsistency within the client and between clients leads to insufficient training of personalized models and hinders collaborative training with other clients. To address this issue, this study proposes a personalized federated learning scheme FedKDF that combines bidirectional knowledge distillation and FIMs, which strikes a balance between personalization and generalization, improves the model's generalization ability and robustness in heterogeneous data environments, and reduces the impact of parameter inconsistency. In addition, experimental results on FishionMNIST, CIFAR-10, and CIFAR-100 show that the FedKDF model proposed in this paper has good performance and robustness in heterogeneous data environments.