A Physics-informed Reinforcement Learning Approach for Multiproduct Pipeline Pump Scheduling Optimization

Pipeline transportation is the primary method for transporting multiproducts, necessitating an effective pump scheduling strategy to ensure safe operation and economic advantages. The working conditions of multiproduct pipelines are complicated, with significant changes in flow and many limitations. Such conditions make it difficult to solve the problem of optimizing the pump schedule. This study developed a discrete-time optimization model for scheduling pumps of multiproduct pipelines and constructed a reinforcement learning (RL) model for training and solving. Then, a physics-informed reinforcement learning (PIRL) framework was proposed to enhance RL training efficiency using state design and action regulation methods. These methods simplified the intricate relationship between state variables and the reward function, mitigated agent exploration in impractical domains, and enhanced the model's training efficiency. This study finally used the trained PIRL model to solve two real-world scheduling problems for multiproduct pumps. Compared to the previous method, the PIRL approach used in this study had lower total costs for the solutions and took 64 and 49 seconds in solving times, which is a 99.7% reduction and a huge improvement in efficiency. The PIRL approach suggested in this study can facilitate the swift solutions of pump scheduling and provide technical support for intelligent pipeline management.