Improved DQN for Path Planning with Classified High-value Prioritized Experience Replay

Aiming at the problems of low sample utilization and slow convergence speed when robots use the DQN algorithm for path planning, an improved DQN algorithm based on classification high-value prioritized experience replay (CHV-PER-DQN) is proposed. First, a dual experience pool is set up to categorize and store experience samples based on immediate reward values. Second, design a high-value prioritized sampling method that combines the state-action value function of the samples with the TD error as a measure of prioritized sampling. Finally, the high-value prioritized experience sampling method and the random uniform sampling method are used to draw experience samples from the two experience pools according to the proportion, respectively, to improve the sampling utilization rate and reduce the impact of the long time-consumption of the traditional prioritized experience replay (PER). Simulation results show that the CHV-PER-DQN algorithm can realize faster convergence speed and better path planning effect than the traditional DQN algorithm and PER-DQN algorithm.