Integrating Task Allocation and Hierarchical Reinforcement Learning for Optimized Cargo Transport Routing

In recent years, there have been growing expectations for the improvement of operational efficiency through the use of artificial intelligence (AI: Artificial Intelligence) in various fields such as healthcare, industry, and services. Additionally, advancements in robotics technology have enabled widespread collaborative operations among multiple autonomous mobile robots. Since the 1980s, there has been active research on improving the efficiency of coordinated actions by multiple autonomous mobile robots using Multi-Agent Robot Systems (MARS) (Cao et al.; 1997, Yan et al.; 2013, Ismail and Sariff; 2019). This study focuses on robots that transport cargo within a warehouse and aims to develop learning methods for effective division of labor among multiple robots. In large and complex environments, agents need to undergo extensive repetitive learning to make appropriate action choices solely through machine learning techniques such as reinforcement learning. Traditional multi-agent methods like QMIX (Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning) (Rashid et al.; 2020) and MADDPG (Multi-Agent Deep Deterministic Policy Gradient) (Lowe et al.; 2017) require substantial computational time for environmental exploration, presenting a significant challenge. Therefore, this paper proposes a two-tiered learning model that separates overall optimization, such as establishing a general procedure for task performance, from individual optimization based on local situational judgments. Additionally, to demonstrate the effectiveness of the proposed method, a simulation system tailored to the target environment is constructed, and simulation analysis based on this system is conducted.