A robust and personalized privacy-preserving approach for adaptive clustered federated distillation

Federated learning (FL) is a promising approach that addresses privacy, and scalability concerns in contrast to traditional centralized methods. Challenges such as personalization and data heterogeneity issues remain critical. Clustered federated learning (CFL) has been proposed as a promising approach to alleviate these issues by establishing specialized global models for sets of similar users. Although CFL enhances adaptability to highly statistically heterogeneous environments, it may suffer from real-time distribution changes due to limitations in fixed cluster configurations. This study presents the robust model of personalized federated distillation (RMPFD), a personalized and privacy-enhanced framework. The RMPFD framework employs an adaptive hierarchical clustering strategy to generate semi-global models by grouping clients with similar data distributions, allowing them to train independently. Meta-learning is used in each cluster to enhance the personalization of the local models and the classification accuracy of the non-independent and Identically distributed (non-IID) data distributions. Experimental evaluations conducted on CIFAR- 10, CIFAR- 100, Fashion-MNIST and Enron email datasets reveal that RMPFD reduces communication overhead by approximately 15% and 20%, compared to Federated Averaging (FedAvg) and other baseline techniques. Moreover, the RMPFD framework improves the convergence rates and classification accuracy, leading to an improvement of over 12% in performance compared to traditional FL methods.