FedAMKD: Adaptive Mutual Knowledge Distillation Federated Learning Approach for Data Quantity-Skewed Heterogeneity

Federated learning enables collaborative training across various clients without data exposure. However, data heterogeneity among clients may degrade system performance. The divergent training goals of servers and clients lead to performance degradation: servers aim for a global model with improved generalization across all data, whereas clients seek to develop private models tailored to their specific local data distributions. This paper introduces a novel federated learning framework named FedAMKD. FedAMKD divides federated learning into two independent entities, a local model tailored to each client's data and a global model for data aggregation and knowledge sharing. A unique aspect of FedAMKD is its adaptive mutual knowledge distillation at the local level, customized for the skewed degree of the client's data quantity. This method achieves the goal of enhancing both local and global model performance, reducing the adverse effects of data quantity-skewed heterogeneity in federated learning. Extensive experiments across diverse datasets validate FedAMKD's success in addressing challenges related to data quantity imbalances in federated learning,