Deep Deterministic Strategy Gradient Method Using Plot Experience Playback

The research on continuous control in reinforcement learning has been a hot topic in recent years. The Deep Deterministic Policy Gradient (DDPG) algorithm performs well in continuous control tasks. DDPG algorithm uses experience replay mechanism to train the network model, in order to further improve the efficiency of experience replay mechanism in the DDPG algorithm, the cumulative reward is used as the transiton classification basis, a Deep Deterministic Policy Gradient with Episodic Experience Replay (EER-DDPG) algorithm is proposed. First of all, the transitions are stored in the unit of episode, and two replay buffers are introduced respectively to classify the transitions according to the cumulative reward. Then, the quality of policy can be improved in network model training period by random samping of the episodes with large cumulative rewards. In the continuous control tasks, this algorithm is verified by experiments, and compared with DDPG algorithm, Trust Region Policy Optimization (TRPO) algorithm and Proximal Policy Optization (PPO) algorithm. The experimental results show that EER-DDPG algorithm has better performance.