AIDPFL: An Adaptive Improvement Approach for Differential Privacy Federated Learning

Federated learning enables participants to train on their local dataset, which solves the privacy preservation problem to some extent. However, attackers can still infer participants' private information from the uploaded parameters. Therefore, adding differential privacy further protects privacy. The key to differential privacy techniques is gradient clipping and gradient noise addition. In gradient clipping, a hyperparameter clipping threshold is introduced. Different combinations of clipping thresholds and learning rates lead to significant variations in accuracy, resulting in increased computational costs when searching for the optimal combination. In gradient noise addition, the gradient gradually decreases with training iterations. Adding fixed noise significantly affects the later stages of the model, leading to a decline in accuracy. In order to solve the above problems, this paper proposes An Adaptive Improvement Approach for Differential Privacy Federated Learning (AIDPFL), specifically (i) adjusting the clipping formula to combine the learning rate and the clipping thresholds under the premise of preserving the gradient information, only the learning rate needs to be adjusted. The clipping threshold size is adjusted in each round. (ii) Dynamically adjust the noise scale according to the gradient change, realize the dynamic decay rate to adjust the noise scale, and ensure that the privacy budget is reasonably allocated in the training process. Our method has higher usability and accuracy than the current primary adaptive differential privacy methods.