AODPFL: An Adaptive Optimization Method for Differentially Private Federated Learning

Federated learning addresses the issues of data silos and privacy to some extent by training models locally on client devices, only uploading model parameters, and aggregating them on the central server. However, attackers can still infer private data information from the uploaded parameters. To solve this issue, differential privacy technology is introduced. The key to differential privacy lies in gradient clipping and noise addition. However, the traditional gradient clipping method often faces the gradient distortion issue and will be ineffective if the noise is large enough. Regarding noise addition, commonly used fixed or fixed decay rate noise scale settings often overlook the characteristics of gradients during training, which might lead to adding improper noise to gradients. To address these issues, we propose an adaptive optimization method for differentially private federated learning (AODPFL). Specifically, we adopt a strategy of clipping the gradient by grouping, effectively reducing gradient distortion. We design an adaptive clipping threshold based on the gradient changes during training to improve model accuracy under large noise conditions. We design a noise scale decay method with a dynamic decay rate to allocate the privacy budget more reasonably and inject appropriate noise into gradients. Experimental results show that compared to other gradient clipping and noise addition methods, our method achieves higher accuracy under the same privacy budget.