Real-Time Scheduling for Multi-Cluster Tools with Residency Constraints by Deep Reinforcement Learning

Multi-cluster tools are extensively utilized in semiconductor manufacturing. As these tools integrate more functionalities and handle increasingly complex processing recipes under numerous constraints, cyclic scheduling strategies become impractical. This poses challenges for non-cyclic scheduling approaches as well. This paper focuses on the non-cyclic scheduling problem of multi-cluster tools with residency time constraints. A generic model is developed to describe dual-armed multi-cluster tools and propose an optimization scheduling algorithm based on Proximal Policy Optimization (PPO). This algorithm is designed to learn effective scheduling policies in environments with small batches of wafers and can scale to handle large batches. Numerical experiments demonstrate that the proposed algorithm enables rapid scheduling, achieving scheduling performance close to optimal and minimizing residency time. Furthermore, comparisons with cyclic scheduling based on implicit enumeration methods show that our approach achieves shorter makespan.