Validation of Synthesa AI, a Large Language Model-Based Screening Tool for Systematic Reviews: Results from Nine Pharmacologic Studies

Systematic review screening underpins the evidence base for pharmacology and drug development but remains burdensome, error-prone, and resource-intensive. Synthesa AI, a large language model (LLM)-based abstract screening tool, was developed to streamline this process by providing a transparent and prompt-driven framework for abstract screening. In this validation study, Synthesa AI was tested across 17 benchmark meta-analyses on nine therapeutic domains relevant to pharmacology and clinical pharmacotherapy. The tool screened 270,626 abstracts retrieved from PubMed and Scopus. Synthesa AI successfully identified all 163 benchmark-included studies, achieving a sensitivity of 100% (95% CI: 97.7%–100.0%) and a specificity of 99.4% (95% CI: 99.37%–99.42%). Importantly, it reduced reviewer workload by 91.7%, with only 1,797 abstracts requiring manual review. Beyond replication, the tool identified 32 additional relevant studies that had been missed in the original reviews, representing a 19.6% increase in evidence yield. These findings highlight the potential of Synthesa AI to enhance pharmacological evidence synthesis by improving the reproducibility and comprehensiveness of systematic reviews used to evaluate drug efficacy, safety, and therapeutic positioning. Synthesa AI represents a transformative solution for living systematic reviews and large-scale evidence integration, offering a rigorous and efficient alternative to traditional human-led screening in pharmacology research.