Transfer learning for linear regression with differential privacy

Abstract Transfer learning, as a machine learning approach to enhance model generalization, has found widespread applications across various domains. However, the risk of privacy leakage during the transfer process remains a crucial consideration. Differential privacy, with its rigorous mathematical foundations, has been proven to offer consistent and robust privacy protection. This study delves into the problem of linear regression transfer learning under differential privacy and, on this basis, proposes a novel strategy incorporating prior information as a constraint to further enhance model performance and stability. In scenarios where the transferable source is known, a two-step transfer learning algorithm incorporating prior information is proposed. This approach leverages prior knowledge to effectively constrain the model parameters, ensuring that the solution space remains reasonable throughout the transfer process. For cases where transferable sources are unknown, a non-algorithmic, cross-validation-based method for transferable source detection is introduced to mitigate adverse impacts stemming from non-informative sources. The effectiveness of the proposed algorithms is validated through simulations and real-world data experiments.