Personalized Federated Knowledge Distillation for Heterogenous Networks

With the proliferation of edge devices, federated learning (FL) has gained significant attention for collaborative model training while preserving data privacy. However, device heterogeneity in FL remains a critical challenge that leads to a reduction in the efficiency of system training. Existing solutions often suffer from rigid task allocation strategies, leading to resource under-utilization or dependencies on specialized hardware that may not be compatible with constrained devices. To address this issue, we propose a personalized federated learning framework to dynamically schedule distributed tasks that integrates model lightweight technologies and knowledge distillation (KD). This approach establishes a computational complexity-to-device capability mapping, enabling dynamic model pruning tailored to individual device resources. This optimization improves task deployment, maximizes resource utilization, and reduces overall training time. Moreover, a dynamic channel distillation method, combined with guided knowledge distillation, is designed to improve the performance of lightweight models. Numerical results demonstrate that our method, compared with baseline methods, optimizes memory usage and accelerates training according to device capabilities while maintaining model accuracy and convergence performance.