RTPbench: A Natural Language to Temporal Logic Translation Benchmark for Multi Robot Heterogeneous Task Planning

For an intelligent large scale robot system, the rea-soning process of each robot should interact with its surrounding environment, cooperate with its nearby comrades and react to the dynamic commands of human manipulator. However, the adaptability of traditional task planning methods is confined to their strong dependencies to environment prior knowledge and robot ability information. To simplify the warping process for dynamic environment and cluster information, our paper proposes a novel translation benchmark from natural language (NL) to temporal logic (TL) for multi robot heterogeneous task planning (RTPbench). The main innovations can be summarized as follows: 1) a texture sensitivity oriented prompt formulation method is proposed to mend the texture deficiency predicament of traditional datasets; 2) a comprehensive prompt generalization strategy is provided to augment temporal logic texture of trans-lation results through landmark based real-world discretization description; 3) an environment information and mobile rule motivated high order texture case embedded prompts are also constructed to expand the reasoning ability of the translator. Extensive experiments on state-of-the-art (SOTA) commercial and open-access large language models (LLMs) has verified the granularity and identification degree of the introduced NL to TL translation benchmark RTPbench.