Federated semi-supervised learning based on feature alignment and knowledge distillation

Introduction Recently, federated learning has been successfully applied in fields related to cyber-physical-social systems (CPSSs), owing to its ability to harness decentralized clients for training a global model while ensuring data privacy. The existing methods encounter two main obstacles, namely, the statistical distribution heterogeneity [non-independent and identically distributed (non-IID)] among clients and the scarcity of labeled data. Methods In this article, we propose a federated semi-supervised learning (FSSL) model under the label-at-server scenario, denoted as FedAlign, which is tailored for distributed cyber-physical-social systems. FedAlign adopts a dual knowledge distillation framework to train the global model. On the client side, FedAlign integrates contrastive learning, knowledge distillation, and pseudo-labeling technology to train local models. The goal is to ensure that global knowledge is not overlooked while enabling clients to learn local knowledge. Meanwhile, on the server side, FedAlign utilizes maximum mean discrepancy to generate a global feature space. Based on the generated feature space, FedAlign employs a knowledge distillation mechanism and supervised learning to aggregate local knowledge and update the global model. Results Two classic datasets, CIFAR-10 and Fashion-MNIST, are used to evaluate the performance of FedAlign. The experimental results demonstrate that FedAlign outperforms traditional federated semi-supervised learning models. Discussion The integration of feature alignment and knowledge enables balancing local knowledge learning and aggregation of global model. As a consequence, FedAlign enhances the adaptability and generalization ability of the global model in CPSSs.