Up-to-date composition and critical appraisal of meta-analyses of comparative studies

Abstract: The number of meta-analyses published every year in abdominal surgery has risen steadily from 118 in 2013 to 247 in 2023. Despite this increased quantity, there are still many common errors committed either in the conception, or the analysis, whether for an original article or in the discussion of other manuscripts to support the results of such meta-analyses. This article aimed to (I) describe the main characteristics of a meta-analysis comparing two procedures, (II) examine the methodology of meta-analyses published between July and December 2023 in the top 10 surgical journals, and (III) assess the impact of the most common errors encountered in the literature. We elaborated seven steps that we think are essential to produce methodologically sound meta-analyses. Errors were identified in meta-analyses published between July 1 and December 31, 2023 in the top 10 visceral surgery journals. We then assessed the impact of these errors on the interpretation of results. We collected 50 meta-analyses (33 full texts and 17 abstracts). Concerning the 33 full-text meta-analyses: randomized trials accounted for 36% while 64% involved observational studies with or without randomized trials. Four of 13 meta-analyses of randomized trials used risk ratios (RR). The random model was applied in 20 (61%) of cases, the fixed model in one (3%) and both models at the same time in 12 (36%) of cases. Heterogeneity was assessed by calculating the I2 in 30 (91%), while no author produced a prediction interval. Publication bias was not investigated in 16 (49%) of cases. Several methodological principles warrant attention: the effect size in meta-analyses of randomized controlled trials (RCTs), should be calculated using the RR while for meta-analyses of observational studies, whether or not associated with RCTs, the odds ratio is most appropriate. Only the random model is suitable for clinical research studies. Heterogeneity should be assessed with the prediction interval; the I2 index is no longer appropriate. These concepts are primordial to underline in future publications of meta-analyses.