A Deep Reinforcement Learning Approach to Optimize Closing Down a Single-Arm Cluster Tool

Cluster tools are widely adopted in semiconductor manufacturing. When a process module fails, a cluster tool must undergo a close-down process, transiting to an idle state. Minimizing the makespan of a close-down process is economically significant in increasing the throughput of a wafer fab. However, the optimization of a close-down process is challenging due to wafer residency time constraints. Existing linear programming models must be constructed when specific processing time and robot activity time parameters are given. To address the issue, we propose a scheduling method based on deep reinforcement learning. For this reinforcement learning algorithm, the specific states, actions, and reward functions are designed for the close-down process to follow the Markov property, and a deep Q-network is adopted to find an optimal scheduling policy. Experimental results demonstrate that, in contrast to traditional scheduling methods, the proposed method achieves an optimal policy and also excels in generalization, enabling adaptive real-time production scheduling.