Selecting the Safest Design in Rare Event Settings

Finitely many simulatable designs are given and we aim to identify the safest one, i.e., that with the smallest probability of catastrophic failure. We consider this problem in a ranking and selection or equivalently the multi-armed-bandit best-arm-identification framework where we aim to identify with high probability the safest design/arm with the lowest probability of failure. To illustrate the rarity structure crisply, we study the problem in an asymptotic regime where the design failure probabilities shrink to zero at varying rates. In this set-up, we consider the well known information theoretic lower bound on sample complexity, and identify the simplifications that arise due to the rarity framework. A key insight is that sample complexity is governed by the rarity of the second safest design. The proposed algorithm is guided by the lower bound, it is intuitive and asymptotically matches the lower bound.