A hybrid deep Q-learning approach to online planning and rescheduling of single-stage multi-product continuous processes

Optimisation-based process scheduling methods serve as a critical response strategy in the process industries, facilitating the coordination of decision-making and execution processes. The efficiency and adaptability of the method are of paramount importance, especially when dealing with frequent modifications to existing decisions caused by uncertainties and unforeseen real-world disturbances, with the goal of achieving substantial financial profitability. Compared to traditional heuristic methods that heavily rely on instance-specific manual customisation and fine-tuning, reinforcement learning (RL) has the advantages of learning from existing experiments and generalising to unknown scenarios, thus automating the process with higher flexibility and adaptability. In this work, we propose an RL-based method that transforms a single-stage multi-product process scheduling problem, originally framed as a mixed-integer linear programming (MILP) problem, into a Markov decision process and trains the RL agent to identify the optimal production sequence. The trained agent is subsequently integrated into a simplified planning and scheduling linear programming (LP) framework to enable efficient decision-making for the re-optimised production sequence and time length. The results show that our proposed learning-based integrated decision-making framework demonstrates strong computational efficiency and adaptability, outperforming both the benchmark random agent and heuristic approach with minimal deviation from the optimal solution achieved by the state-of-the-art MILP solvers.