Privacy-Preserving and Top-K Sparsified Federated Learning with Low Communication Overhead

Federated learning addresses the issue of data silo in machine learning. However, in practical applications, it still encounters challenges such as privacy leakage and communication bottleneck. Previous studies have proposed two main technologies to these challenges: secure aggregation to preserve privacy and Top-k gradient sparsification to reduce communication overhead, respectively. However, for both privacy preservation and communication efficiency, combining these two technologies results in compatibility issues and additional privacy leakage. In this paper, we propose a secure aggregation protocol with Top-k sparsification to achieve secure and efficient federated learning. We employ a differential privacy perturbation mechanism to protect Top-k features, thus preventing client's privacy leakage. Additionally, we design a sparse communication graph to ensure compatibility between secure aggregation and Top-k sparsification perturbed by differential privacy. We prove that our protocol protects Top-k features and conduct extensive experiments to evaluate its performance, which shows a significant reduction in the communication overhead compared to traditional secure aggregation protocols.