A Comprehensive Monte Carlo Simulation Study on Multiple Comparison Methods after ANOVA

Multiple comparison methods are applied to control the type I error rate at the nominal level. In this study, we investigate the performance of multiple comparison methods after analysis of variance (ANOVA) is implemented under different conditions. We include Bonferroni, Holm, Hochberg, Hommel, Benjamini-Hochberg (BH), and Benjamini-Yekutieli (BY) correction methods. Monte-Carlo simulation study is applied to assess their performances under different patterns, including sample size and group number combinations. Wide inferences are drawn on considered methods, and suggestions are provided for selecting appropriate methods. Moreover, the methods are implemented on three different types of real-life data sets to emphasize the importance of these correction methods in the research.