Automatic pipeline layout optimization based on deep reinforcement learning

Automatic pipeline layout optimization in complex environment is a highly challenging task. However, the traditional pipeline layout optimization method is not enough to deal with the complex changes of the environment and the requirements of multi-pipeline layout process. Therefore, this paper transforms this pipeline layout optimization problem into an MDP and proposes a pipeline layout optimization algorithm based on deep reinforcement learning. Specifically, we introduce a design method of layers in pipeline space to reduce the computational burden caused by an increase in the number of pipelines and combine with the discrete Soft Actor-Critic algorithm to improve the exploration efficiency of pipeline space, thereby to avoid falling into the local optimum of pipeline layout prematurely. Then, we verify the proposed pipeline layout optimization algorithm through simulation experiments. Experimental results show that the automatic pipeline layout optimization is significantly promoted in the complex, and various environments and the craft requirements of multi-pipeline layout are also satisfied.