An Order-Free End-to-End Multi-AGV Scheduling Method Based on Safe Deep Q-Network and Asynchronous Coordination

Abstract Automated Guided Vehicle (AGV) scheduling is critical to enhance the resource utilization and ensure the timely task completion in the shop floors. Some prior works deal with the AGV scheduling problem through heuristic algorithms. Due to the challenge of accurate mathematical modeling, deep reinforcement learning-based methods have been developed by recent works. However, they mostly require the dispatching orders to be generated in advance, which can be difficult for some end users. The computational efficiency and scalability also have been rarely explored. To tackle these issues, this paper proposes an end-to-end, lightweight and scalable scheduling method relying only on high-level descriptions of the ultimate goal. A Markov decision process formulation is specially designed to encode the studied problem involving extra buffer zone and pallet. Then a lightweight deep Q-network is developed to fuse the multi-source state information and learn optimal policy. Considering the physical constraints, a shielding mechanism is further employed to ensure the accessibility of the learnt policy. Finally, a multi-AGV coordination algorithm is designed to realize joint dispatching operations of multiple AGVs. Experiments on a real shop floor of a large plant show the superiority of the proposed method in terms of the total makespan as well as the scalability towards prevalent changes in the shop floor layout.