Model-Heterogeneous Federated Learning With Bidirectional Knowledge Distillation

Federated Learning (FL) faces significant challenges in addressing the system heterogeneity of edge devices, which often exhibit diverse computational and resource constraints. To overcome these limitations, we propose FedBid, a novel model-heterogeneous FL framework based on bidirectional, data-free knowledge distillation. By synthesizing data on the server and integrating forward-backward knowledge distillation with group aggregation, FedBid allows devices to adopt different models tailored to their resources and enabling efficient knowledge sharing across diverse model architectures without relying on public datasets. In addition, we further introduce progressive learning and generator perturbation strategies to enhance the efficiency of bidirectional knowledge distillation. Extensive experiments on benchmark datasets (Fashion-MNIST, CIFAR-10) and a real-world dataset (FLAIR) under varied data distributions demonstrate that FedBid achieves superior model performance than the state-of-the-art methods while reducing computational overhead on resource-constrained clients.