Estimated Tardiness-Based Reinforcement Learning Solution to Repeatable Job-Shop Scheduling Problems

Intelligent manufacturing promises to revolutionize production processes, and it is expected to enhance efficiency and productivity. In this paper, we study the job-shop scheduling problem, which is a key enabler to the realization of intelligent manufacturing. Compared to previous studies, the proposed solution tackles generalized configurations by allowing the repetition of jobs and by assuming the limited capacity of machines and a sequence of operations constituting a job. The proposed solution is fully controlled by the trained reinforcement learning agent that finds the optimal match between the job and machine to schedule. In addition, by introducing the expected tardiness (ETD) metric, the agent can enhance the scheduling performance while effectively handling dynamic action space changes with the action masking technique. To effectively adapt to the particular manufacturing site, we propose a novel training approach that utilizes the average order arrival distribution learned from the historical logs. Such data-driven optimization can train an agent that effectively captures the general and site-specific characteristics of job arrivals, leading to improved generalization performance and a finely tuned model, respectively. To validate the proposed approach, we implement a custom environment with which extensive performance evaluation and comparison are carried out. The evaluation results show that the proposed approach can outperform the conventional heuristic priority dispatching rules under the desired performance criteria such as total tardiness and the total manufacturing cost. To be specific, in terms of the total cost metric, the proposed approach outperforms the considered approaches by 31.69% on average. In addition, the use of ETD can enhance the performance of the conventional approaches by 27.74% on average.