Structured Expert Elicitation of Dependence Between River Tributaries Using Nonparametric Bayesian Networks

In absence of sufficient data, structured expert judgment is a suitable method to estimate uncertain quantities. While such methods are well established for individual variables, eliciting their dependence in a structured manner is a less explored field of research. We tested the performance of experts in constructing and quantifying a nonparametric Bayesian network, describing the correlation between river tributary discharges. Specialized software was provided to assist the experts. Expert performance was investigated using the dependence calibration score (a correlation matrix distance metric) and the likelihood of the joint distribution. Desirable properties of the dependence calibration score were investigated theoretically. Individual expert judgments were combined based on performance into a group opinion aka decision maker. All experts were able to create and quantify a correlation matrix between 10 variables that resembled the correlations between observed discharges well. The decision makers performed similarly to the best expert. Based on the metrics investigated, it mattered little which expert opinions and with what weight were combined in a decision maker. This is partly because all experts performed well. Adding a bad performing expert increased the positive effect of performance-based weighting, underscoring the importance of developing scoring rules for dependence elicitation. The overall results are promising: Aided by specialized graphical software, the experts in this study were able to quickly create and quantify dependence structures.