RLGBS: Reinforcement Learning-Guided Beam Search for process optimization in a paper machine dryer section

Paper drying is responsible for over two-thirds of energy consumption in the U.S. pulp and paper industry, presenting significant potential for energy savings through optimization of process parameters. Current approaches often assume fixed operating conditions, neglecting dynamic ambient and process variations that limit achievable savings and real-world applicability. To this end, we develop a physics-based simulation environment for a paper machine dryer section and propose a reinforcement learning (RL) framework to minimize overall energy consumption by optimizing drying process parameters under diverse operating conditions. To mitigate overdrying and numerical instabilities caused by suboptimal local RL actions, we introduce Reinforcement Learning-Guided Beam Search (RLGBS), which explores multiple action sequences in parallel using beam search. Instead of making step-by-step decisions, RLGBS prioritizes solutions based on cumulative probability, reducing the impact of individual suboptimal actions. Experiments demonstrate that RLGBS achieves consistent energy savings under unseen operating conditions not encountered during training, outperforming conventional RL methods. While validated in drying optimization, this framework is broadly applicable to other RL-based industrial process control problems. • Reinforcement Learning-Guided Beam Search (RLGBS) is developed for process optimization. • A simulation environment is created to enable fast interactions with a physics-based drying model. • RLGBS is shown to improve the quality of generated actions in paper drying optimization. • RLGBS effectively addresses various operating conditions and process uncertainties.