LLM-Driven Testing for Autonomous Driving Scenarios

In this paper, we explore the potential of leveraging Large Language Models (LLMs) for automated test generation based on free-form textual descriptions in area of automotive. As outcome, we implement a prototype and evaluate the proposed approach on autonomous driving feature scenarios in CARLA open-source simulation environment. Two pre-trained LLMs are taken into account for comparative evaluation: GPT-4 and Llama3. According to the achieved results, GPT-4 outperforms Llama3, while the presented approach speeds-up the process of testing (more than 10 times) and reduces cognitive load thanks to automated code generation and adoption of flexible simulation environment for quick evaluation.