Adaptive Exploration Deep Q-Network for AGV Path Planning in Dynamic Environments Using 3D Convolutional Neural Networks

This paper addresses the AGV path planning problem in dynamic environments using deep reinforcement learning. An improved deep Q-network (DQN), AEDQN (Adaptive Exploration DQN), is proposed, incorporating an adaptive epsilon and a heuristic exploration strategy to balance exploration and exploitation while avoiding deadlocks and accelerating convergence. For dynamic path planning, the traditional DQN struggles with predicting the motion of dynamic obstacles, so a 3D-CNN-AEDQN algorithm is introduced, using a 3D convolutional neural networks for temporal image data. Additionally, a global dynamic path planning approach is proposed, combining deep reinforcement learning with local environment adaptation to improve obstacle avoidance. Simulation results show that the AGV successfully avoids the three dynamic obstacles and reaches the destination. The algorithm converges at 2600 iterations and the test success rate is 82.5%, which demonstrate the effectiveness of 3D-CNN-AEDQN and the adaptive strategy in complex environments with both static and dynamic obstacles.