A Reinforcement Learning and Hyper-Heuristic Guided Hierarchical Learning Algorithm for Petroleum Production Scheduling Problem

The petroleum production scheduling problem within the context of intelligent manufacturing involves the integrated linkage of multiple factories located in different regions throughout the full-process production, as well as the sequencing and scheduling of multiple production tasks. Starting from the actual production process, this paper abstracts a distributed mixed no-wait flow shop scheduling problem with multiple heterogeneous constraints (DMNWFSP-MHC), considering the technological advancement and quantity of machines in distributed factories, and differences in transportation distances. Firstly, a mathematical model for DMNWFSP-MHC is established. Secondly, a reinforcement learning and hyper-heuristic guided hierarchical learning algorithm (RLHHL) is proposed to address the complex scheduling problem. In RLHHL, a priority-based initialization rule is adopted to improve the initial populations. A reinforcement learning-driven hierarchical algorithm framework is developed to achieve intelligent configuration of multiple operators under various characteristics. A dynamic reward based on convergence rate feedback the search behaviors, assisting the population in making decisions. Multiple neighborhood search operators based on problem knowledge are designed to form a hierarchical search space. Finally, an elite sequence-guided local search strategy is improved to guide the population in quickly finding the optimal solutions. The RLHHL is compared with four state-of-the-art algorithms for DMNWFSP-MHC, and the effectiveness in settling the DMNWFSP-MHC is verified through the test experiments with different scales.