A Synthetic Data-enhanced Differential Privacy Reinforcement Learning Policy Optimization Method

Offline reinforcement learning (offline RL) has achieved success in various domains in recently year, which helping researcher deploy the policy into the real-word engineering tasks. But there still a challenge in protecting the privacy of the sensitive offline datasets, introducing privacy protect technology into those sensitive datasets is a feasible method to handle this challenge. The recent research prove that different privacy (DP) is an efficient way to introduce DP mechanism into offline RL algorithm, but the DP mechanism could also decrease the policy performance seriously. To address this issue, in this work we purpose Differential Privacy Impact Policy Performance Compensate Buffer (DPCB), a diffusion-based approach to provide high quality dataset for training a well-performing policy with privacy protection. We show that DPCB has a generalization capacity combinate with different offline RL algorithm. Furthermore, we compare DPCB with other offline RL different privacy protect algorithm in privacy protect effectiveness and policy performance, the experiment indicates that DPCB is an effective method to compensate the impact of policy performance and policy oscillation brought by DP mechanism.