Fidelity assessment of synthetic images with multi-criteria combination under adverse weather conditions

With the development of AI-based perception algorithms using cameras, access to large and representative datasets is crucial. For autonomous driving systems, it is essential to use road context datasets covering the entire Operating Design Domain, including various road configurations, driving scenarios, and weather conditions. In this context, it is imperative to propose mechanisms and metrics that allow quantifying the fidelity and the level of representativeness of these simulated datasets, in order to evaluate and validate their usability for training and evaluation stages. In this paper, we propose an objective and multi-modal approach allowing to calculate 4 scores representing several aspects of the synthetic image fidelity. These scores address local, global and statistical texture analysis. In addition, a multi-criteria approach, based on evidence theory, is proposed to merge these scores to obtain a final global score. The result is the generation of a global score along with the uncertainty and conflict quantification. This method has been applied on a large number of real and virtual datasets in different weather conditions (clear, rainy, foggy). The initial results are promising and confirm the interest and the relevance of this method.