Using Large Language Models to Assess the Consistency of Randomized Controlled Trials on AI Interventions With CONSORT-AI: Cross-Sectional Survey
GPT-4 variants demonstrate strong performance in assessing the consistency of RCTs with CONSORT-AI. Nonetheless, refining the prompts could enhance the precision and consistency of the outcomes. While AI tools like GPT-4 variants are valuable, they are not yet fully autonomous in addressing complex and nuanced tasks such as adherence to CONSORT-AI standards. Therefore, integrating AI with higher levels of human supervision and expertise will be crucial to ensuring more reliable and efficient evaluations, ultimately advancing the quality of medical research.
Authors
- Yaolong Chen (ORCID: https://orcid.org/0000-0002-7338-4418)
- James Zou (ORCID: https://orcid.org/0000-0001-8880-4764)
- Yanfang Ma (ORCID: https://orcid.org/0000-0001-6772-2460)
- Zeming Li (ORCID: https://orcid.org/0009-0002-0895-4354)
- Fengxian Chen
- Xufei Luo (ORCID: https://orcid.org/0000-0003-0811-6326)
- Zhaoxiang Bian (ORCID: https://orcid.org/0000-0001-6206-1958)
- Lu Zhang (ORCID: https://orcid.org/0000-0002-2794-7371)
- Qi Wang (ORCID: https://orcid.org/0000-0002-5060-5978)
- Zhenhua Yang
- Bingyi Wang (ORCID: https://orcid.org/0009-0004-8834-893X)
- Long Ge (ORCID: https://orcid.org/0000-0002-3555-1107)
Institutions
- Hong Kong Baptist University (HK)
- Chinese University of Hong Kong (HK)
- Chinese Academy of Medical Sciences & Peking Union Medical College (CN)
- Impact (CA)
- Lanzhou University (CN)
- University of Hong Kong (HK)
- Stanford University (US)
- McMaster University (CA)
Publication Details
- Journal
- Journal of Medical Internet Research
- Published
- 2025-09-26
- DOI
- https://doi.org/10.2196/72412
- Citations
- 1
- Primary Topic
- Artificial Intelligence in Healthcare and Education
- Type
- article
- Field-Weighted Citation Impact
- 0.65