HMA-PPO for Integrated Production and Preventive Maintenance Scheduling in Distributed Flexible Job Shops with Job Arrivals

Abstract With the rapid advancement of distributed manufacturing, distributed flexible job shops have become an important production paradigm in high-end equipment manufacturing. However, dynamic job arrivals and preventive maintenance requirements arising from continuous equipment degradation create strong coupling between production scheduling and maintenance decisions, making multi-objective optimization particularly challenging. This study investigates the integrated optimization of production scheduling and preventive maintenance in distributed flexible job shops with dynamic job arrivals, with the objectives of minimizing makespan, total tardiness, and total energy consumption. To solve this problem, a hierarchical multi-agent proximal policy optimization approach, termed HMA-PPO, is proposed. The method establishes a collaborative decision-making framework that involves production agents and maintenance agents, enabling the joint optimization of production scheduling and preventive maintenance. A multi-objective learning mechanism that combines NSGA-II-generated reference solutions with dynamic weights derived from the entropy weight method is further developed to alleviate reward sparsity and improve multi-objective trade-off performance. In addition, a discrete-event simulation environment is developed in FlexSim to model and evaluate the collaborative scheduling process under dynamic job arrivals. Experimental results on generated test instances show that HMA-PPO consistently outperforms PPO, SAC, DDQN, and DQN in terms of convergence quality and Pareto-set diversity, as evaluated using IGD, Nd, C(A,*), and NR.