On limiting behaviors of stepwise multiple testing procedures

Abstract Stepwise multiple testing procedures have attracted several statisticians for decades and are also quite popular with statistics users because of their technical simplicity. The Bonferroni procedure has been one of the earliest and most prominent testing rules for controlling the familywise error rate (FWER). A recent article established that the FWER for the Bonferroni method asymptotically (i.e., when the number of hypotheses becomes arbitrarily large) approaches zero under any positively equicorrelated multivariate normal framework. However, similar results for the limiting behaviors of FWER of general stepwise procedures are nonexistent. The present work addresses this gap in a unified manner by elucidating that, under the multivariate normal setups with some correlation structures, the probability of rejecting one or more null hypotheses approaches zero asymptotically for any step-down procedure. Consequently, the FWER and power of the step-down procedures also tend to be asymptotically zero. We also establish similar limiting zero results on FWER of other popular multiple testing rules, e.g., Hochberg’s and Hommel’s procedures. It turns out that, within our chosen asymptotic framework, the Benjamini–Hochberg method can hold the FWER at a strictly positive level asymptotically under the equicorrelated normality.