Multi-products production control and human-autonomous truck distribution planning with the use of collaborative multi-agent reinforcement learning

In the era of Industry 5.0, the integration of human-centric values with advanced automation is pivotal. This paper addresses a complex distribution planning problem that necessitates the collaboration of human distributors and autonomous trucks within a manufacturing setting. We formulate the problem as a Partially Observable Markov Decision Process (POMDP) and propose a hierarchical multi-agent reinforcement learning (MARL) framework that unifies production control with distributor planning. At the upper level, two different agents determine production quantities for both preordered and on-time products, while concurrently planning appropriate distribution channels based on fluctuating market conditions. At the lower level, a decentralized execution strategy enables human and robotic distributors to dynamically fulfill customer demands. Key to this approach is a communication-based centralized learning scheme that leverages differentiable inter-agent learning to coordinate decisions, ensuring that autonomous operations do not undermine the human workforce, which is a core tenet of Industry 5.0. Computational experiments demonstrate that the proposed MARL framework outperforms traditional heuristic algorithms and conventional RL methods, achieving superior performance through improved coordination and adaptability. The results highlight not only the efficiency gains from automation but also the essential balance maintained between technological advancement and human involvement.