An Exploration Method for Deep Reinforcement Learning Based on State Prioritized Replay

The exploration of environment is a vital foundation for deep reinforcement learning, and the exploration efficiency is directly related to the model training efficiency and interactive costs. In this paper, we propose an exploration method for deep reinforcement learning based on state prioritized replay, which realizes the automatic adjustment of the starting point of exploration. This paper analyzes the effect of the exploration algorithm on various algorithms and tasks, and verifies the effectiveness and robustness of the method in reducing the number of exploration steps and training time through multiple sets of hyperparameter experiments. Experiments show that compared with simple exploration, the method in this paper can reduce the number of exploration steps by 21.2% and 45.5%, respectively, and optimize the time indicators by 9.3% and 45.4% when applied to discrete tasks such as CartPole-v0. On continuous tasks such as Pendulum-v0, the number of exploration steps can be reduced by 41.9% and 8.3%, respectively, and the time indicators can be optimized by 40.7% and 10.1%.