Principal Angle-Based Clustered Federated Learning With Local Differential Privacy for Heterogeneous Data

Local differential privacy federated learning has attracted wide attention because it can solve the problem of data islands without damaging data privacy, but it faces data heterogeneity problems on the client side. Existing solutions commonly cluster clients with similar data distributions into the same training groups by leveraging clients’ model parameters, thus mitigating the impact of data heterogeneity on federated learning performance. However, model parameters, as an indirect representation for client data, often fail to accurately reflect the true data distribution, resulting in inaccurate client groupings. Furthermore, these solutions perturb high-dimensional parameter vectors dimension-by-dimension to protect client data privacy, which introduces substantial LDP noise that significantly further compromises the client grouping accuracy. To tackle these challenges, we propose PCFed-LDP, a privacy-preserving clustered federated learning framework that improves the federated learning performance while protecting client data and satisfying LDP in heterogeneous environments. Specifically, we introduce a client clustering method based on geometric properties of client data subspaces, which conducts label grouping-based principal angle analysis on client data subspaces to accurately capture the similarities in client data distributions, thereby enabling precise client grouping. To reduce the amount of introduced LDP noise, we design an adaptive noise addition method that utilizes the Haar wavelet technique to decouple the relationship between the noise amount and vector dimensionality, and employs noise error minimization strategy-based vector segmentation to inject LDP noise with finer granularity. Theoretical analysis and experiments on real datasets demonstrate that our solution not only satisfies the constraints of local differential privacy but also outperforms state-of-the-art methods.