A Hybrid Method Based on Multi-Agent Reinforcement Learning and Integer Programming for Dynamic Slab Design Problems in Steel Industry

This paper investigates a dynamic slab design problem in the steel industry, where order demands arrive dynamically during a given period. Slabs are the raw materials for producing order plates demanded by customers — slabs are first rolled in a rolling mill to create desired mother plates, and then the mother plates are cut into order plates. The dynamic nature of orders, along with practical considerations regarding rolling methods and nonlinear size constraints, distinguish our problem from existing ones. The goal is to determine slab design schemes to fulfill order demands for the period. However, the stochastic nature of dynamic production and the inherent complexity of slab design present significant challenges in the efficient solution. To address these challenges, we formulate a Partially Observable Markov Decision Process (POMDP) and propose a hybrid method (MARLIP) based on multi-agent reinforcement learning (MARL) and integer programming. The MARL component determines the order plate set, the slab type, and the rolling method involved in a single slab design. Then, an integer programming model is constructed to optimize the arrangement of the order plates based on MARL’s decisions. To further refine the MARLIP approach, we introduce dual factors and a soft boundary into the MARL and propose a two-core-based dynamic programming method to solve the integer programming model. Experimental results demonstrate our algorithm’s superior performance compared to several competitive reinforcement learning and mathematical programming methods in the dynamic slab design problem.