Dynamic Path Planning Scheme for OHT in AMHS Based on Map Information Double Deep Q-Network

AMHSs (Automated Material Handling Systems) are widely used in major Fabs (semiconductor fabrication plants). The OHT in an AMHS is responsible for handling the FOUP (Front Opening Unified Pod) within the Fabs. Due to the unidirectional track, the movement path of the OHT aims to avoid congested areas caused by operations or malfunctions as much as possible, to improve the overall FOUP handling efficiency. To do so, we propose a dynamic path planning method, MI-DDQN (Map Information Double Deep Q-Network), driven by deep reinforcement learning and based on map information. Firstly, we design and establish a map information state space model based on the core elements of the OHT path planning in the AMHS. Then, we design an OHT motion simulator to simulate the position coordinate transformation of the OHT, providing real-time coordinate update data for the OHT during the algorithm training process. We design a deep reinforcement learning algorithm structure based on map information model and a convolutional neural network model structure and use the algorithm to train the network model. Finally, the designed task generation module and OHT motion simulator are used to randomly generate the starting position and task position of the OHT during the training process to enhance the richness of the data. The addition of a “fault” OHT verifies the method’s ability to plan routes in complex road conditions such as congestion that may occur at any time.