Is it all relative? Proposing the use of marginal effects for meta-analysis of binary outcomes in criminology and criminal justice

Meta-analyses of criminal justice interventions often analyze outcomes that are binary in nature. Summary effect sizes in these analyses are commonly represented using odds or risk ratios, which attempt to create comparability across studies by expressing treatment effects as being relative to the baseline outcome in the control group. One cost of this benefit, however, is that binary effect sizes are no longer interpreted in terms of probability, the natural metric of the dependent variable. Increasingly, scholars have recommended the use of marginal effects to express the results of binary outcome models on the probability scale. However, these methods have rarely been applied to meta-analysis, particularly in the field of criminal justice. In this paper I propose and demonstrate the use of marginal effects for meta-analysis of binary outcomes in criminology and criminal justice. In doing so, I describe estimation strategies for the summary marginal effect, multiple methods for calculating the standard error of the marginal effect, and present the results of simulations testing the coverage and width of the confidence intervals produced by each method. Ultimately, I argue that the summary marginal effect should be reported, not as a replacement for common effect size metrics, but as a supplement to those metrics. I suggest that there is value in understanding binary outcomes on the probability scale, and that reporting both the average standardized and marginal effect size across studies provides a broader, more policy-relevant interpretation of meta-analytic findings. • Effect size metrics for binary outcomes limit intuitive interpretation of results. • Summary marginal effects, or absolute effect sizes, can be estimated during meta-analysis. • Delta-method standard errors are a valid approach to calculating the uncertainty surrounding the summary marginal effect. • Summary marginal effects can supplement the interpretation of traditional effect size estimates.