Integrating Deep Model-Based Learning With Modular State-Based Stackelberg Games for Self-Optimizing Distributed Production Systems

This article introduces a novel integration of deep model-based learning with modular state-based Stackelberg games (Mod-SbSG) for distributed self-optimization in manufacturing systems, using a sample-efficient approach. Model-free Mod-SbSG requires frequent interactions with real systems to find optimal solutions, which can be costly, time-consuming, and risky in industrial settings. Prior studies handled this by using digital representations to train Mod-SbSG players, but accurate representations are often difficult to develop. Hence, our framework replaces digital representations with deep learning methods that learn system dynamics, optimize policies within Mod-SbSG, and reduce real-world interactions. The method includes two main steps: 1) designing deep learning models to predict system dynamics and 2) training Mod-SbSG players in virtual environments. We evaluate single- and multistep predictors and demonstrate network reuse for transfer learning in adaptable systems, which reduces real system interactions by 77.78% in a laboratory testbed industrial control scenario.