Comparison of interval estimation for extreme event proportions based on exact, approximate and Bayesian approaches

In the statistical literature, several large comparative studies have been conducted on the confidence intervals of proportions. However, few comparative studies have focused on the confidence intervals for extreme event proportions (e.g. zero-failure or all-success events) calculated using exact, approximate, and Bayesian methods. In this study, an extensive simulation with three-level nesting, comprising 980 million parameter space points, was designed based on the relationship between the binomial and beta distributions. After thoroughly examining three evaluation criteria – interval width, coverage, and oscillation – the exact method is recommended for estimating confidence intervals for extreme event proportions. The rationale for this recommendation is explained from four perspectives: exact vs. approximate methods, frequentist vs Bayesian methods, average coverage vs coverage oscillation, and the representativeness of the evaluated methods. Three real-world case studies further demonstrate the validity of this recommendation.