Differentially Private Federated Learning Model with Second-Order Optimization

Federated Learning (FL) enables diverse devices to collaborate on model training without sharing raw data, raising privacy concerns due to the potential exploitation of uploaded gradients by adversaries. The Differentially Private Federated Learning (DPFL) framework introduces noise to local parameters before transmission to the Parameter Server to protect user privacy but may cause performance degradation, particularly in over-parameterized models. To tackle these challenges, we propose a novel algorithm that utilizes second-order optimization to enhance model convergence speed within the DPFL framework. By incorporating second-order information, our approach aims to mitigate performance degradation while maintaining user privacy. We conduct theoretical analysis on convergence and sensitivity of local updates, ensuring privacy guarantees using Rényi Differential Privacy. Experimental results on EMNIST, CIFAR-10, and CIFAR-100 datasets demonstrate the superior accuracy and privacy protection of our proposed method compared to existing approaches.