SavviDriver: model-based framework for game-based testing of autonomous vehicles in diverse multi-agent traffic scenarios

Abstract Autonomous vehicles (AVs) must operate safely in the face of uncertainty, including those induced by human behaviors (i.e., external human drivers). Specifically, AVs must exhibit safe responses when encountering previously unseen behaviors from human drivers with different driving styles. For example, aggressive drivers may cut off other vehicles to merge into a lane, or distracted drivers may fail to respond to changing road conditions. A key challenge is how to assess the onboard AV decision-making capabilities to detect and mitigate those potentially unsafe scenarios due to one or more external human-operated vehicles. We observe that AVs and other vehicles on the roadway may share common functional objectives (e.g., to navigate to a given target destination), but otherwise may be motivated by different non-functional objectives, such as safety, minimizing transport time, minimizing fuel consumption, etc. This paper introduces a modular and composable model- and game-based testing framework to enable an AV developer to operationally assess the robustness of an AV in response to human-based uncertainty. Specifically, this work uses goal models to declaratively specify functional and non-functional objectives of vehicles (both the AV under study and those representing external human-operated vehicles) to inform the game-based testing environment that incorporates real-world traffic infrastructure data. We demonstrate the model-based capabilities of our game-based testing approach on a number of scenarios based on real-world traffic accident data involving human drivers.