Efficient Collaboration of Multiple Robots Based on Deep Reinforcement Learning in Intelligent Manufacturing Scenarios

The collaborative multi-robot system (CMRS) is a strong solution for intelligent manufacturing (IM) scenarios, which can greatly improve production efficiency. However, with traditional scheduling methods, the systems are usually specially designed for particular scenarios with pre-set robots, machines and tasks, since there are few general solutions capable for various and complex scenarios in the industry. In this paper, we propose a general, end-to-end, and scalable solution for efficient collaboration of heterogeneous multi-robot systems. First, we establish a general description model for typical IM scenarios with complex workflow and extensible constraints. Then, we design a heterogeneous extension of multi-agent reinforcement learning (MARL) approach to facilitate efficient production in heterogeneous CMRS with transporting robots and different types of processing machines, where the collaborations at both schedule level and control level are considered. In the end, a visualized simulation environment for IM production is developed and a case scenario is used to evaluate the performance of our approach. The results show that the proposed policy significantly improves the production performance in terms of both the in-time completion ratio and average order delay, compared with popular rule-based methods and the homogeneous MARL method.