Personalized Differential Privacy for Ridge Regression Under Output Perturbation

ABSTRACT The increased application of machine learning (ML) in sensitive domains requires protecting the training data through privacy frameworks, such as differential privacy (DP). Traditional DP enforces a uniform privacy level , which bounds the maximum privacy loss that each data point in the dataset is allowed to incur. However, this one‐size‐fits‐all approach fails to reflect the reality that individuals often have different privacy expectations‐depending on factors such as the sensitivity of their data, legal requirements, or personal risk tolerance. As a result, uniform DP can either overprotect some data (hurting utility) or underprotect others (violating privacy needs). In this work, we introduce our Personalized‐DP Output Perturbation method (PDP‐OP) that enables us to train Ridge regression models with individual per data point privacy levels, in the central privacy model. Our method relies on a technique known as output perturbation, that was introduced by Chaudhuri and Monteleoni, augmented with re‐weighting data points according to their privacy levels. Additionally, we provide rigorous privacy proofs and accuracy guarantees for PDP‐OP. Thereby, our work distinguishes itself by providing theoretical accuracy guarantees in personalized DP settings in ML, whereas similar previous work only provided empirical evaluations. To demonstrate how our theoretical bounds hold in practice, we evaluate PDP‐OP on synthetic and real datasets and with diverse privacy distributions. We show that by enabling each data point to specify their own privacy requirement, we can significantly improve the privacy‐accuracy trade‐offs compared to non‐personalized DP. Finally, we also show that PDP‐OP outperforms the personalized privacy techniques introduced by Jorgensen et al. that rely on subsampling as opposed to reweighting.