Federated Learning Based on Knowledge Distillation and Network Pruning

Aiming at the problem that the training effectiveness of deep learning models is closely related to labeled training data, a federated learning framework, FedPrune, is proposed with high training efficiency and accuracy; a knowledge distillation method is adopted and furthermore a network pruning scheme is used to generate a suitable local student model from a global teacher model, which realizes the lightweight intelligent algorithms to be executed efficiently in the resource-constrained devices; comparison experiments are conducted with the 2 local solutions and 2 global solutions on CIFAR10, CIFAR100 datasets for comparison experiments, FedPrune has higher accuracy and shorter training time.