Toward Universal Embodied Planning in Scalable Heterogeneous Field Robots Collaboration and Control

ABSTRACT Multi‐robot systems offer substantial enhancements in efficiency, scalability, robustness, and flexibility for executing complex tasks through collaborative efforts. However, existing methodologies are constrained by their lack of generalizability, the need for extensive modeling, and most importantly, limitations in their applicability in complex scenarios. This paper presents a novel approach to multi‐robot task planning and coordination, introducing a comprehensive pipeline encompassing data generation, supervised fine‐tuning, and rigorous error analysis using the Multi‐Robot collaboration Error Diagnostic (MRED) metrics. Bridging the gap between natural language commands and physical groundings in robot collaboration tasks, we present MultiPlan : the first data set specifically designed for LLM fine‐tuning. The MultiPlan data set encompasses 100 distinct indoor and outdoor scenarios, ranging from office to garden. Experiments underscore the efficacy of the proposed methodology, including comparative analyses against state‐of‐the‐art LLMs and generalization studies on previously unseen tasks. Results reveal that the fine‐tuned model achieves a 24.8% relative improvement over the GPT‐4 model in addressing complex multi‐robot planning scenarios. We also conducted field evaluations in both office and urban settings to demonstrate the deployment performance of the proposed method. These results demonstrate the model's superior capabilities in task decomposition, error management, and adaptation to novel contexts.