Assessing Microscopic Traffic Simulation Confidence Level Through Expert-Weighted AHP and Belief Theory: An Application to CAV Testing

Abstract This paper addresses the challenge of assessing experts' confidence in simulation tools for reproducing safety-related indicators in traffic scenario-based testing. While simulation offers major advantages in terms of safety, cost, and repeatability, its outputs do not equally reflect reality across all traffic situations. The objective is therefore to identify which scenarios can be considered high confidence in simulation. The proposed approach relies on several Analytic Hierarchy Process (AHP) approaches combined with an expert survey on microscopic traffic simulation. Scenarios are represented through ontology-based attributes, and confidence weights are assigned through pairwise comparisons. Three fusion strategies are compared: classical AHP, weighted AHP based on expertise level, and AHP combined with belief theory. The results show that accounting for expertise improves the confidence scores and leads to more consistent rankings of scenarios. Dynamic traffic and environmental conditions are generally associated with lower confidence than static road-related attributes. Scenario score distributions and Gaussian mixture analysis further reveal distinct groups of low-, medium-, and high-confidence scenarios. These results provide a decision-support framework for distinguishing scenarios associated with higher expert-assigned confidence from those that may require real-world testing.