Methodological Adherence, Reporting Completeness, and Risk of Bias in AI-Assisted Systematic Reviews: A Matched Meta-Research Study

Background: Artificial intelligence (AI) is increasingly used to accelerate systematic reviews that inform clinical practice and policy. However, evidence on the methodological adherence, reporting completeness, and risk of bias of completed AI–assisted reviews is limited. Objective: To compare methodological adherence, reporting completeness, and risk of bias between published systematic reviews with reported AI assistance and matched reviews with no reported AI use, and to assess AI–use transparency. Methods: We conducted a matched–cohort meta–research study of English–language, peer–reviewed health systematic reviews. AI–assisted reviews published during 2023—2025 were matched 1:2 to reviews with no reported AI use using individualized MEDLINE searches targeting publication timing, clinical domain, review type, and meta-analysis status. When more than two eligible comparators were available, two were randomly selected. AI assistance was defined as reported use during searching, screening, data extraction, risk–of–bias assessment, evidence synthesis, or substantive drafting. Full texts and supplementary materials were examined to classify reported AI use. The primary outcome was a prespecified modified 13–item AMSTAR–2 methodological–adherence score (range 0 — 13; noninferiority margin &−1 point). Secondary outcomes included PRISMA 2020 reporting completeness and ROBIS risk of bias; AI–use transparency was assessed descriptively. Adapted 13–item AMSTAR–2 confidence categories were summarized descriptively, while critical– and noncritical–domain adherence was examined in exploratory adjusted analyses. Multivariable regression models adjusted for journal impact tier, funding reporting, formal review–group affiliation, review–team size, and review–question type, using cluster–robust standard errors by matched set. Sensitivity analyses included matched–set fixed–effects models. The study was registered on the Open Science Framework (DOI: 10.17605/OSF.IO/Q5JRW). Results: The cohort included 231 systematic reviews: 77 AI–assisted and 154 with no reported AI use. Mean 13–item AMSTAR–2 scores were 7.58 and 9.63, respectively. In multivariable–adjusted analyses, AI–assisted reviews scored 1.64 points lower (95% CI &−2.37 to &− 0.91); noninferiority was not demonstrated. PRISMA adherence was estimated to be 4.7 percentage points lower among AI-assisted reviews in primary adjusted analyses (95% CI &−8.9 to &−0.6) and 6.4 points lower in matched–set fixed–effects analyses (95% CI &−10.7 to &−2.1). Among reviews with assessable ROBIS ratings, 75.0% of AI-assisted reviews and 65.6% of comparators were rated low risk (adjusted difference +17.8 percentage points; 95% CI 2.7 to 33.0). Among AI-assisted reviews, model version or date was reported in 42.9%, prompts or parameters in 55.8%, and data-governance or ethics considerations in 9.1%. Conclusions: Reviews reporting AI assistance had lower methodological–adherence scores and had less complete reporting, despite more favorable risk–of–bias ratings. These findings provide an early warning that AI–assisted systematic reviews may require stronger safeguards, human verification, and transparency to support trust.