AutoMeta: LLM Agent-based Automated Meta-Analysis in Public Health
Background: Meta-analysis supports public health decision-making, yet conducting and updating reviews remain time- and labour-intensive. Although large language model (LLM) agent systems have been proposed to automate literature reviews, heterogeneous exposure definitions, effect measures and covariate-adjustment models pose challenges for public health evidence synthesis. Methods: We present AutoMeta, an LLM-based agentic workflow for public health meta-analysis that uses stage-specific instructions to guide planning and tool use across literature search, screening, data extraction and statistical synthesis, with structured outputs linking each stage. To address the heterogeneity of public health evidence, it expands exposure terminology, structures eligibility assessments, links extracted estimates to their analytical context, and uses tested functions for effect-size transformation and synthesis. For systematic evaluation, we constructed AutoMetaBench from 27 systematic reviews spanning environmental health, nutrition and health behaviours, encompassing 421 included-study records, 4164 data fields and 95 effect units. We used this benchmark to assess performance at each stage and end to end. Results: AutoMeta improved evidence coverage and synthesis accuracy while supporting rapid end-to-end execution. In stage-specific evaluations, it outperformed the respective comparators across all four stages, achieving 93.1% search recall, 94.8% recall among the top 50 ranked records, and 93.6% data extraction accuracy. Agreement with reference pooled effects and 95% confidence intervals reached 96.8% and 95.8%, respectively. Across 27 end-to-end tasks, AutoMeta reproduced reference pooled effects and both 95% confidence interval bounds within 5% relative error in 85.2% of tasks, compared with 7.4% for the single-agent comparator, while taking an average of only 26 minutes per task. Conclusions: AutoMeta demonstrates substantial advantages in evidence synthesis and statistical analysis for public health meta-analysis. Its LLM-based agentic workflow improves synthesis efficiency and accuracy, while AutoMetaBench provides a dedicated benchmark for the systematic evaluation of automated methods. These contributions advance automated public health evidence synthesis and support timely evidence updates.
Authors
- Xingyue Tian (ORCID: https://orcid.org/0000-0003-4144-7004)
- Tong Xia (ORCID: https://orcid.org/0009-0001-8425-8433)
- Yuying Huang
- Ke Zhu
- Yamin Sun
Institutions
- Tsinghua University (CN)
Publication Details
- Journal
- Research Square
- Published
- 2026-11-20
- DOI
- https://doi.org/10.21203/rs.3.rs-11389292/v1
- Primary Topic
- Meta-analysis and systematic reviews
- Type
- preprint