AutoMeta: LLM Agent-based Automated Meta-Analysis in Public Health

Background: Meta-analysis supports public health decision-making, yet conducting and updating reviews remain time- and labour-intensive. Although large language model (LLM) agent systems have been proposed to automate literature reviews, heterogeneous exposure definitions, effect measures and covariate-adjustment models pose challenges for public health evidence synthesis. Methods: We present AutoMeta, an LLM-based agentic workflow for public health meta-analysis that uses stage-specific instructions to guide planning and tool use across literature search, screening, data extraction and statistical synthesis, with structured outputs linking each stage. To address the heterogeneity of public health evidence, it expands exposure terminology, structures eligibility assessments, links extracted estimates to their analytical context, and uses tested functions for effect-size transformation and synthesis. For systematic evaluation, we constructed AutoMetaBench from 27 systematic reviews spanning environmental health, nutrition and health behaviours, encompassing 421 included-study records, 4164 data fields and 95 effect units. We used this benchmark to assess performance at each stage and end to end. Results: AutoMeta improved evidence coverage and synthesis accuracy while supporting rapid end-to-end execution. In stage-specific evaluations, it outperformed the respective comparators across all four stages, achieving 93.1% search recall, 94.8% recall among the top 50 ranked records, and 93.6% data extraction accuracy. Agreement with reference pooled effects and 95% confidence intervals reached 96.8% and 95.8%, respectively. Across 27 end-to-end tasks, AutoMeta reproduced reference pooled effects and both 95% confidence interval bounds within 5% relative error in 85.2% of tasks, compared with 7.4% for the single-agent comparator, while taking an average of only 26 minutes per task. Conclusions: AutoMeta demonstrates substantial advantages in evidence synthesis and statistical analysis for public health meta-analysis. Its LLM-based agentic workflow improves synthesis efficiency and accuracy, while AutoMetaBench provides a dedicated benchmark for the systematic evaluation of automated methods. These contributions advance automated public health evidence synthesis and support timely evidence updates.

Authors

Institutions

Publication Details

Journal
Research Square
Published
2026-11-20
DOI
https://doi.org/10.21203/rs.3.rs-11389292/v1
Primary Topic
Meta-analysis and systematic reviews
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

AutoMeta: LLM Agent-based Automated Meta-Analysis in Public Health

Xingyue Tian, Tong Xia, Yuying Huang, Ke Zhu et al.
5 links
Research Square
Meta-analysis and systematic reviews
preprint

AutoMeta: LLM Agent-based Automated Meta-Analysis in Public Health

Xingyue Tian, Tong Xia, Yuying Huang, Ke Zhu, Yamin Sun
preprint en

Abstract

Background: Meta-analysis supports public health decision-making, yet conducting and updating reviews remain time- and labour-intensive. Although large language model (LLM) agent systems have been proposed to automate literature reviews, heterogeneous exposure definitions, effect measures and covariate-adjustment models pose challenges for public health evidence synthesis. Methods: We present AutoMeta, an LLM-based agentic workflow for public health meta-analysis that uses stage-specific instructions to guide planning and tool use across literature search, screening, data extraction and statistical synthesis, with structured outputs linking each stage. To address the heterogeneity of public health evidence, it expands exposure terminology, structures eligibility assessments, links extracted estimates to their analytical context, and uses tested functions for effect-size transformation and synthesis. For systematic evaluation, we constructed AutoMetaBench from 27 systematic reviews spanning environmental health, nutrition and health behaviours, encompassing 421 included-study records, 4164 data fields and 95 effect units. We used this benchmark to assess performance at each stage and end to end. Results: AutoMeta improved evidence coverage and synthesis accuracy while supporting rapid end-to-end execution. In stage-specific evaluations, it outperformed the respective comparators across all four stages, achieving 93.1% search recall, 94.8% recall among the top 50 ranked records, and 93.6% data extraction accuracy. Agreement with reference pooled effects and 95% confidence intervals reached 96.8% and 95.8%, respectively. Across 27 end-to-end tasks, AutoMeta reproduced reference pooled effects and both 95% confidence interval bounds within 5% relative error in 85.2% of tasks, compared with 7.4% for the single-agent comparator, while taking an average of only 26 minutes per task. Conclusions: AutoMeta demonstrates substantial advantages in evidence synthesis and statistical analysis for public health meta-analysis. Its LLM-based agentic workflow improves synthesis efficiency and accuracy, while AutoMetaBench provides a dedicated benchmark for the systematic evaluation of automated methods. These contributions advance automated public health evidence synthesis and support timely evidence updates.

Research Square
Tsinghua University (CN)
Good health and well-being
Meta-analysis and systematic reviews
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon