Personalized Decentralized Federated Learning: A Privacy-Enhanced and Byzantine-Resilient Approach

Personalized decentralized federated learning (PDFL) has emerged recently to address the problem of single point of failure and data heterogeneity in traditional centralized federated learning. However, existing works on PDFL still have two challenges that urgently need to be solved. First, model updates exposed by point-to-point communication during collaborative training in PDFL may disclose sensitive information about clients. Second, the distributed structure makes PDFL vulnerable to Byzantine attacks, which can disrupt the network by introducing poisoned data or faulty behaviors. In this article, we propose a privacy-enhanced and Byzantine-resilient approach to effectively address the dual challenges of privacy and security in PDFL. In particular, each client is required to build a unique critical parameter index set by evaluating the importance of its model parameters and broadcasting it to neighbors. To improve Byzantine resilience, we propose a novel weight allocation scheme based on the critical parameter index set for clients to alleviate the negative impact of Byzantine neighbors in the model aggregation. To enhance privacy protection while boosting personalization, we combine with model decoupling and execute a clipping-robust personalized local training for each client to achieve user-level differential privacy. We finally conduct exhaustive experiments on FEMNIST, SVHN, and CIFAR10 datasets and various settings. Experimental results demonstrate that compared to five state-of-the-art baselines, our proposed method achieves excellent performance with user-level differential privacy guarantee in PDFL and implements additionally superior Byzantine robustness in adversarial settings.