Dynamic Differentially Private Personalized Federated Learning with Hessian Diagonal Information

Personalized federated learning with differential privacy is a promising solution to address the challenges posed by non-independent and identically distributed (non-IID) data distributions and the risk of privacy leakage. However, prevailing personalized approaches rely on static partitioning of model components, which inherently lacks flexibility. In addition, excessive noise injected into critical model parameters during the later stages of training can significantly hinder model convergence. To address the two above difficulties, we propose to leverage the diagonal of the Hessian as an effective metric for quantifying parameter sensitivity, thereby estimating the information content of each model parameter. Specifically, we use the Hessian diagonal information to measure the information content of local model parameters at each client; parameters with high information value are retained locally as personalized model components, while the remaining parameters are uploaded to the server for aggregation as part of the shared model. Furthermore, to accelerate convergence, we progressively increase the proportion of personalized parameters by retaining more high-information parameters locally during training. Finally, we evaluate the proposed method on image classification tasks using CIFAR-10, CIFAR-100, and EMNIST datasets, and observe significant performance improvements under the same privacy budget.