Large language models for systematic review tasks compared with Cochrane reviews: a protocol for evaluating accuracy, agreement, and time efficiency

Large language models (LLMs) show promise for accelerating systematic review (SR) stages, yet comprehensive comparative evaluations against validated standards remain lacking. To compare the accuracy, agreement, and time efficiency of four LLMs (GPT, Claude, Gemini, and DeepSeek) across five SR stages—search strategy formulation, deduplication, article selection, risk of bias assessment, and evidence synthesis—against published Cochrane Reviews as the reference standard. Comparative study with randomized assignment using a Williams Latin square design. Twenty five Cochrane SRs of interventions (2020–2025) will be evaluated by 15 researchers stratified by experience. Prompts will be developed using the 4D framework; the underlying process model will be validated via Delphi consensus, and resulting prompt wording via expert content review. Outcomes include recall, sensitivity, Cohen’s kappa, and execution time. Ethical exemption will be requested. Results will be disseminated through five peer-reviewed publications.