LLM Assistant for heterogeneous multi-robot system dynamic task planning

Multi-robot systems find good implantation in construction applications. The combination of the skills of the robots creates systems with high flexibility. However the combination of these skills is not enough to ensure complex task completion. Task planning needs to face the dynamic behaviors of construction environments. In this paper, a heterogeneous multi-robot system for construction tasks is presented. The system is composed by three robots that present complementary skills and is commanded through natural language requirements, imposing low training requirements for the user. Task planning is managed by a large language model assistant, which receives information about the environment and the state of the system along with the user request. With this information, the assistant provides the execution plan for the robots in a dynamic environment. The system is tested through different situations with real-world tasks, involving extensive interaction with the environment and dynamic planning. Environment recognition, tool manipulation, or obstacle removal are some of the tasks presented in the system.