Automatically Generating Content for Testing Autonomous Vehicles from User Descriptions

Testing autonomous vehicles (AV) software, which is currently done using simulations, requires the availability of various content, such as terrains and maps, to instantiate relevant scenarios. Manually generating such content is time-consuming, and current approaches for procedural content generation struggle to handle user requirements. Consequently, the limited availability of content strongly affects AV testing effectiveness. To address this problem, we present RoadGPT, the first generative AI approach that generates focused scenarios by translating user requirements in natural language into three-dimensional road models. RoadGPT leverages OpenAI’s foundational large language model (LLM) ChatGPT to interpret user descriptions and the physically accurate driving simulation BeamNG.tech to generate the corresponding driving simulations. Our initial evaluation, which includes a focused user study with experts in the AV testing domain, confirmed the ability of RoadGPT to generate roads matching user-defined descriptions and highlighted venues for future improvements. We believe that RoadGPT can become an essential component in AV testing and can be extended to create other relevant testing environments, such as parking spaces.