pFedMMD:Personalized Federated Learning via Meta-Learning and Model Disentanglement

Traditional Federated Learning (FL) allows participants to train a shared global model on a distributed client. However, changes in the distribution between real-life non-IID datasets often pose a threat to such a solution where all clients use the same model, and heterogeneity between clients often hinders optimal convergence and generalization performance. Whereas, the purpose of personalized FL is to alleviate this problem by each client autonomously training a personalized model that conforms to its own local data. However, a model trained with a small amount of local data is not ideal. The training goal of meta-learning is to train a model, which only needs a small amount of data and iterative training to quickly adapt to new tasks, and the main factor of the effect of MAML is feature reuse, that is, better performance can be achieved by removing the head of the model, and feature quality is crucial to the success of meta-learning algorithms. In this paper, we study a federated meta-learning of a personalized variant, improve it for the characteristics of heterogeneous data distribution, and prepare a personalized model composed of the initial model base layer + local model personalization layer for federated training for finite round federated learning. We show the experimental results of the proposed method for simulated a variety of non-independent identical distribution (Non-IID) data scenarios on four datasets of federated learning: CIFAR-10, CIFAR-100, MNIST, and Fashion-MNIST through data partitioning.