Probabilistic or possibilistic expert knowledge modeling? Dunning-Kruger curve helps to choose!

<ns3:p>Background Interval estimates are a common way to express uncertain knowledge of experts. To model them and aggregate multiple judgments, both the probability and possibility theories are applicable. Previous studies have shown that the performances of the aggregated distributions obtained by these two approaches are similar on average; however, there is a lack of works investigating how we can establish a preference between them in certain cases. Methods The distribution of expert-based interval estimates on the latent Dunning-Kruger curve, i.e., the correlation between their accuracy (estimation error) and confidence/precision (interval width) was determined. The judgments were modelled using both probabilistic and possibilistic approaches, and the estimation errors of the obtained aggregated distributions were compared, described by an advantage score. Its dependence on the confidence-accuracy interdependence of expert judgments was investigated involving estimates for multiple variables. Results Interval estimates of ten experts regarding nine properties of a manual waste sorting system were analyzed including feed composition, product purity and yield. The results show that there is a strong correlation between the confidence-accuracy interdependence of expert judgments and the advantage score. When narrower interval estimates imply greater accuracy, the probabilistic approach is preferable. However, in the reverse case, the possibilistic method yields better results. Conclusions Our basic intuition is that narrower interval estimates are more accurate than wider ones. In this case, the probabilistic approach for modeling expert knowledge is appropriate. However, as the Dunning-Kruger effect highlights, sometimes its reverse is true; then, the possibilistic approach tends to be more suitable as it does not amplify the effect of narrow estimates. The results show that the choice between the two concepts can be based on the correlation trend between the accuracy and precision of judgments that could be deduced, e.g., from the composition of the expert group.</ns3:p>