Think Different: Adaptive Privacy Budget Allocation for Privacy-Preserving Machine Learning

Privacy preservation in the learning phase of machine learning poses considerable challenges. Two main approaches are commonly used to address these challenges: adding noise to machine learning model parameters to improve accuracy, and using noisy data during the learning process to enhance privacy. Recently, the Scalable Unified Privacy-preserving Machine Learning framework (SUPM) has emerged as a promising solution, effectively balancing privacy and accuracy by integrating privacy protection across the stages of dimension reduction, training, and testing. This paper introduces a novel method that optimizes privacy budget allocation by assigning budgets to various attributes based on their relevance to the target attribute. This approach improves accuracy while minimizing the reduction of relevant attributes. When incorporated into SUPM, our algorithm enhances both accuracy and privacy preservation. We evaluate its performance using logistic regression and support vector machines as the underlying machine learning models, demonstrating its effectiveness in retaining accuracy and maintaining attribute integrity. Additionally, we compare our approach with other uneven privacy budget allocation methods, such as Markov-kRR, confirming the superiority of our technique. We further examine the specific conditions under which our method proves particularly effective for certain datasets.