Evaluation of reliability, repeatability, and confidence of ChatGPT for screening, monitoring, and treatment of interstitial lung disease in patients with systemic autoimmune rheumatic diseases
GPT-4o mini and GPT-4o demonstrated stable reliability across all three attempts, whereas GPT-4 did not. The repeatability of GPT-4o tended to perform better than GPT-4o mini, although this difference was not statistically significant. Additionally, GPT-4o exhibited a higher tendency toward overconfidence compared to GPT-4o mini. Overall, the GPT-4o models performed most effectively in managing SARD-ILD but may exhibit overconfidence in certain scenarios.
Authors
- Yingwei Zhang (ORCID: https://orcid.org/0000-0002-0950-2627)
- Chuanjun Xu
- Xiang Qu (ORCID: https://orcid.org/0000-0003-1094-8922)
- Ruyi Zou (ORCID: https://orcid.org/0000-0002-2332-6658)
- Yanghong Wang
- Minjie Lin (ORCID: https://orcid.org/0000-0003-4529-9475)
- Biyun Xu (ORCID: https://orcid.org/0000-0002-5610-2677)
Institutions
- Nanjing University of Chinese Medicine (CN)
- Xuzhou Medical College (CN)
- Nanjing Drum Tower Hospital (CN)
- Second People’s Hospital of Huai’an (CN)
- Fourth Affiliated Hospital of Guangxi Medical University (CN)
Publication Details
- Journal
- Digital Health
- Published
- 2025-05-01
- DOI
- https://doi.org/10.1177/20552076251384233
- Primary Topic
- Ultrasound and Hyperthermia Applications
- Type
- article
- Field-Weighted Citation Impact
- 0.00