Towards effective privacy preservation in federated learning with automatic gradient clipping and gradient transformation perturbation

Abstract Differential privacy can effectively help federated learning resist privacy attacks from various parties. However, existing approaches that use differential privacy for privacy protection greatly decrease the model performance of federated learning, especially in scenarios with complex model structures and large parameters. In this paper, we propose a novel privacy preservation scheme for federated learning that combines automatic gradient clipping and gradient transformation perturbation. Our approach primarily reduces the impact of differential privacy on federated learning from two aspects. Firstly, we efficiently control the gradient sensitivity by using automatic gradient clipping instead of traditional threshold clipping. Secondly, we utilize the space transformation technique to alleviate the dramatic accuracy degradation of the model caused by the insertion noise. Extensive experiments on various benchmark datasets demonstrate that our approach achieves a good trade-off between data privacy and effectiveness under the same privacy budget.