Dynamic scheduling of human-robot collaboration with deep reinforcement learning

Abstract Dynamic scheduling in Human-Robot Collaboration (HRC) is crucial for realizing the human-centric manufacturing goals of Industry 5.0, yet it faces significant challenges from uncertainties, particularly those stemming from human factors, which impede real-time optimization. This paper proposes a novel dynamic scheduling model for HRC assembly environments, leveraging Deep Reinforcement Learning (DRL). The model utilizes a Graph Neural Network (GNN) to embed complex system states, including operator characteristics and task assignments (human-only, robot-only, and human-robot collaborative). A scheduling policy, trained using the Proximal Policy Optimization (PPO) algorithm, is developed to dispatch tasks dynamically with the objective of minimizing makespan while adapting to uncertain operational events. The efficacy of the proposed approach was evaluated through numerical experiments based on a realistic lithium battery pack assembly scenario, considering various HRC configurations and human performance variability modeled using skewed distributions. Comparative analysis against traditional metaheuristic algorithms (Genetic Algorithms with crisp, fuzzy, and neutrosophic number representations) and OR-Tools demonstrated the DRL model’s superior performance. It consistently achieved significantly lower makespan and improved scheduling stability (lower deviation in makespan), particularly in complex multi-human, multi-robot configurations and in managing process uncertainties. The findings highlight the potential of DRL to provide robust and adaptable solutions for dynamic HRC scheduling, thereby enhancing the efficiency and reliability of collaborative manufacturing systems.