Design of a Multi-robot Coordination System based on Functional Expressions using Large Language Models

A system is expected to facilitate coordination among multiple construction machines or robots, enabling them to adaptively perform various tasks in disaster sites and unknown environments. Prior research has generally adopted a model-based approach to designing cooperative behavior. However, it is difficult to adapt to environments and scenarios that cannot be predicted by the model. In recent years, it has been reported that a robot equipped with foundation models can adapt to unknown (open) environments and unpredictable situations. However, there has been little discussion on foundation models for multiple robot systems; a flow that cooperatively handles unexpected events does not exist. In this paper, we propose the system flow that enables multiple robots to adaptively coordinate to unforeseen scenarios based on the functional expressions of each other and environment understanding utilizing GPT-4 and GPT-4V. Through experimentation, we verify that the proposed flow is able to adapt to an unforeseen environment, particularly path obstruction via robot experiments. Furthermore, we examine the validity of the proposed flow by varying the robots’ functional expressions and sensor information for the environment.