Deep reinforcement learning for assembly robot end effector scheduling

Efficient scheduling of robot end effectors in assembly production is a crucial challenge in flexible and adaptive manufacturing, particularly when considering sequence-dependent setup times (SDST). This paper introduces a deep reinforcement learning (DRL) framework designed to optimize scheduling in robotic assembly systems. We address the problem using a proximal policy optimization (PPO) agent, trained to minimize the overall makespan through a dense reward structure. The scheduling task is formulated as a flexible job shop scheduling problem (FJSSP), with SDST modeled through the removal and installation times of end effectors, and minimum lag times ensuring realistic task sequencing. We benchmark our DRL-PPO approach against classical dispatching rules, a genetic algorithm, and a constraint programming-based solver, demonstrating its competitiveness on small to medium-sized problem instances. Although the DRL-PPO agent shows promising performance relative to classical methods, its efficacy diminishes for larger problem instances, highlighting the need for further architectural enhancements. Our findings lay the groundwork for the integration of DRL into complex scheduling environments, offering a foundation for future innovations in flexible and adaptive manufacturing systems.