A comparison of combined <i>p</i>-value functions for meta-analysis

<i>P</i>-value functions are modern statistical tools that unify effect estimation and hypothesis testing and can provide alternative point and interval estimates compared to standard meta-analysis methods, using any of the many <i>p</i>-value combination procedures available (Xie et al., 2011, JASA). We provide a systematic comparison of different combination procedures, both from a theoretical perspective and through simulation. We show that many prominent <i>p</i>-value combination methods (e.g. Fisher's method) are not invariant to the orientation of the underlying one-sided <i>p</i>-values. Only Edgington's method, a lesser-known combination method based on the sum of <i>p</i>-values, is orientation-invariant and still provides confidence intervals not restricted to be symmetric around the point estimate. Adjustments for heterogeneity can also be made and results from a simulation study indicate that Edgington's method can compete with more standard meta-analytic methods.