How does AI compare to the experts in a Delphi setting: simulating medical consensus with large language models
LLM-based Delphi methods demonstrated high clinical consensus closely aligned with human expert decisions. LLMs effectively simulated structured human-like deliberative reasoning, though they tended to adopt more guideline-driven and conservative positions. However, the use of commercial LLM platforms limits control over model parameters that may affect reproducibility. While the current study suggests that LLMs hold promise as complementary tools in medical consensus-building processes, further research addressing parameter optimization is warranted.
Authors
- Songchang Shi (ORCID: https://orcid.org/0000-0001-5793-569X)
- Abdelrahman Ali Nimeri (ORCID: https://orcid.org/0000-0001-7844-163X)
- Ahram Han (ORCID: https://orcid.org/0000-0001-7111-7095)
- Ali Tavakkoli (ORCID: https://orcid.org/0000-0002-7430-1848)
- Eric G. Sheu (ORCID: https://orcid.org/0000-0003-2707-1768)
- Dongjae Jeon
- Young Suk Park (ORCID: https://orcid.org/0000-0002-9069-4821)
Institutions
- Brigham and Women's Hospital (US)
- Fujian Medical University (CN)
- Seoul National University Hospital (KR)
- Seoul National University Bundang Hospital (KR)
- Fujian Provincial Hospital (CN)
Publication Details
- Journal
- International Journal of Surgery
- Published
- 2025-10-15
- DOI
- https://doi.org/10.1097/js9.0000000000003631
- Citations
- 2
- Primary Topic
- Artificial Intelligence in Healthcare and Education
- Type
- article
- Field-Weighted Citation Impact
- 1.31