LLM for Generating Simulation Inputs to Evaluate Path Planning Algorithms

In computer science and robotics research that focuses on algorithm designs, simulation is oftentimes the first step in validating the developed algorithms. However, simulation inputs need to be designed as close as possible to real-world scenarios so that a particular algorithm will perform equally well in simulation as in real-world testing. Designing credible simulation inputs is time-consuming and requires a fair amount of human labor, arguably due to the lack of an efficient way that streamlines the design process while having the capability to provide enough variations. In this study, we present the first-ever exploratory effort to use a Large Language Model (LLM) to facilitate generating simulation inputs. Specifically, we introduce two distributed Multi-Agent Path Finding (MAPF) algorithms and then utilize an LLM to generate variations of warehouse layouts to be used to validate our algorithms. We detail how to effectively prompt the LLM for simulated warehouse designs and compare algorithm performance on both human and LLM-created layouts. Our experimental results show that the LLM-generated layouts find the same algorithm performance trends as inputs designed by humans but require much less time to create, highlighting the LLM's potential to speed up simulation environment generation for algorithm testing.