FedKD-MR: Enabling Heterogeneous Federated Learning through Multi-Round Knowledge Distillation

In heterogeneous IoT deployments, federated learning is challenged not only by distributed, non-IID data but also by the possibility that participating devices may rely on incompatible model architectures. Under these conditions, classical parameter aggregation becomes difficult to apply because client weights cannot be meaningfully combined. FedKD-MR addresses this problem by shifting coordination from the parameter space to the output space: clients exchange logits computed on a shared, public reference dataset, enabling collaboration across heterogeneous models under a common input-label interface. Its main contribution is to organize this output-space exchange into a multi-round process in which consensus logits are repeatedly aggregated, redistributed, and reused during local distillation and supervised fine-tuning. We evaluate FedKD-MR on CIFAR-10 and CIFAR-100 under heterogeneous, non-IID client configurations, including cases in which parameter aggregation cannot be applied to the full client population. The results show gains over isolated local training. We further evaluate the method on UCI-HAR to examine a sensor-oriented setting and the role of the public anchor dataset. This analysis shows that public-set size, class coverage, and distributional quality directly affect the usefulness of consensus logits. Overall, the results support the use of multi-round logit reuse as a practical alternative for coordinating heterogeneous IoT clients when shared model weights are not feasible.