A critical evaluation of the metastatistical approach to extremes with focus on bias, uncertainty bands, and goodness-of-fit test

Abstract Classical extreme value analysis (EVA) often provides large uncertainties on estimated return levels due to the limited amounts of data available. Marani and Ignaccolo (Adv Water Resour 79:121–126, 2015. https://doi.org/10.1016/j.advwatres.2015.03.001 ) aim to overcome this by the metastatistical extreme value (MEV) approach. Here extremes are treated as large ordinary events described by one common, known distribution, and therefore a much larger pool of data is available for estimation. They performed Monte Carlo simulations with synthetic Weibull-distributed rainfall series and showed that the MEV approach gives unbiased estimates of extremes with a smaller uncertainty than classical EVA does. However, the MEV approach neglects that many complex physical mechanisms influence rainfall and other hydrological processes. This means that the tail behavior of the distribution cannot necessarily be inferred from the ordinary events. We therefore replicated their work but added new Monte Carlo experiments to study the classical EVA and the MEV methodologies with a slightly perturbed tail of the underlying distribution. When applying the MEV approach, i.e. fitting a Weibull distribution to the perturbed Weibull series, we obtained consistently negatively biased estimates with underestimated uncertainty bands. In contrast, classical EVA also produced unbiased estimates here. Finally, we showed that goodness-of-fit tests are not able to provide guidance on whether MEV can provide unbiased and confident return levels. Further Monte Carlo simulations showed that these conclusions seem to be quite general and not dependent on the specific distribution. Consequently, the MEV approach may have limitations that make it less suitable for providing reliable return levels in real-world applications.