Analyzing the Sensitivity of a Scenario Selection Process for Simulation‐Based Safety Testing

ABSTRACT Automated driving systems (ADS) require exhaustive safety testing. Simulation‐based safety testing is a cost‐effective method for conducting safety testing. However, an ADS may be exposed to a large number of driving scenarios in the real world. It is not feasible to test every scenario in the simulator. We previously introduced the simulation‐based safety testing scenario selection (SSTSS) process for simulation‐based safety testing of ADS. In order to be useful, the SSTSS process must reflect input changes correctly in its outputs. To investigate this, we vary two key inputs: (i) the operational design domain (ODD) and (ii) the car crash dataset to analyze the sensitivity of the SSTSS process. We selected two different ODDs: one comprehensive, covering diverse urban driving conditions, and one more restricted, defining a controlled environment. We then use two different car crash datasets, the US and EU datasets, as well as subsets of these datasets. We compare the total number of selected scenarios and the correlation between the priority of selected scenarios when different ODDs and datasets are used. Our results show that the SSTSS process shows appropriate sensitivity to the ODD and car crash dataset used, allowing testers to focus on relevant, non‐redundant test scenarios.