An Adaptive Model Difference Clipping Method for Differentially Private Federated Learning

Federated learning (FL) is a privacy-preserving distributed machine learning framework that allows collaborative model training among multiple clients without disclosing their raw training data. Despite data staying localized in FL, clients are still susceptible to privacy threats as their private or sensitive information can be inferred from their shared model updates or gradients. To address this issue, differential privacy (DP) techniques have been widely employed in FL by introducing random noises to obfuscate clients’ shared model parameters, thus alleviating privacy leakage. However, the introduction of DP to FL often significantly compromises the accuracy of model training. In this work, we propose a novel differentially private federated learning method to strike a desirable balance between model training accuracy and client privacy. The core idea of our method is that in each round, each client first clips the model difference between their updated model and the received global model and introduces random noises to the clipped model difference, which is then uploaded to the server. In particular, we design an adaptive clipping strategy where the clipping bound is dynamically adjusted. We conduct extensive experimental evaluations of our proposed method, and the results show that it achieves superior model accuracy under the same level of privacy protection.