CCTest: Critical Configuration Testing for Autonomous Driving Systems

Scenario-based testing is the dominant approach for validating autonomous driving systems (ADS) through simulation. However, generating critical scenarios that effectively uncover real defects in ADS while managing the complexity of the generation remains a challenge. This paper introduces CCTest, a test framework that generates critical test data using simple scenarios decomposed from complex ones and provides qualitative analysis of the scenarios. CCTest ensures that any safety violation observed indicates a defect in the tested autopilot by creating situations where each vehicle is controlled by the autopilot under test and has a feasible safety policy. Evaluations have shown that CCTest effectively revealed major defects in Apollo, Autoware, and the built-in autopilots in Carla and LGSVL simulators. CCTest also outperforms existing tools in efficiently uncovering real defects in autopilots.