Multi-Agent Soft Actor-Critic for Production Rate Control

The evolution of manufacturing systems towards distributed architectures presents unique challenges in optimising production rates across multiple machines in various locations. To address this, effective coordination of production rates at each stage is crucial to ensure the production capacity of these distributed systems, minimise production-related expenses, and account for the impact of traffic conditions on inter-factory material transportation. In this paper, we explore the use of Multi-Agent Soft Actor-Critic (MASAC), a relatively under-explored Multi-Agent Reinforcement Learning (MARL) algorithm, for controlling production rates in a distributed manufacturing environment. Unlike traditional methods that address the credit assignment problem through value decomposition, this framework leverages a multi-headed Q-network to effectively identify and evaluate individual contributions of agents to the global reward. The MASAC algorithm is evaluated against MADDPG and DDQN algorithms, with simulation results showing that MASAC achieves high production output while significantly reducing energy, storage and computational costs. In addition, we compare the performance of MASAC with MAPPO.