Why Black-Box Bayesian Safety Assessment of Autonomous Vehicles Is Problematic and What Can Be Done About It

This paper deals with the Bayesian safety assessment of autonomous vehicles (AV) conducted via driving AVs on the public roads, often referred to as “driving to safety.” A key safety measure is the probability of catastrophic failure (i.e., a road accident) per mile of driving (pfm), assumed a random variable. We argue that a Bayesian prediction based on a univariate (“black-box”) probabilistic model has an intrinsic deficiency: it cannot accommodate the variation of pfm due to changing road conditions, which in turn may affect significantly the predicted pfm and may lead to optimistic predictions. A multivariate probabilistic model is developed to overcome this limitation of the univariate model. Using a set of contrived examples the predictions of the multivariate model are compared with those derived with univariate models. Our results provide an intriguing insight that even when AV driving does not lead to accidents at all, the pfm predictions with the multivariate model may be more pessimistic than the assumed prior, and those derived with a black-box model, including the predictions using the recently developed “conservative Bayesian inference”. The multivariate Bayesian safety assessment can be applied to autonomous vehicles and to other complex intelligent systems such as robots, UAVs, etc., where the operating conditions vary.