While GPT-3.5 is unable to pass the Physician Licensing Exam in Taiwan, GPT-4 successfully meets the criteria
ChatGPT-3.5 showed proficiency in basic medical sciences but was less reliable in clinical medicine, whereas ChatGPT-4 demonstrated strong capabilities in both areas. However, their proficiency varied across different specialties. The type of question had minimal impact on performance. This study highlights the potential of AI models in medical education and non-English languages examination and the need for cautious and informed implementation in educational settings due to variability across specialties.
Authors
- Tzeng‐Ji Chen (ORCID: https://orcid.org/0000-0002-8350-0232)
- Yu‐Chun Chen (ORCID: https://orcid.org/0000-0002-4124-1949)
- Ming-Hwai Lin (ORCID: https://orcid.org/0000-0001-6825-6692)
- Hsiao‐Ting Chang (ORCID: https://orcid.org/0000-0003-0421-8499)
- Kuan-Chen Lin (ORCID: https://orcid.org/0000-0003-2922-3673)
- Tsung-An Chen
Institutions
- National Yang Ming Chiao Tung University (TW)
- National Chung Hsing University (TW)
- Taipei Veterans General Hospital (TW)
- Yuli Veterans Hospital (TW)
Publication Details
- Journal
- Journal of the Chinese Medical Association
- Published
- 2025-03-14
- DOI
- https://doi.org/10.1097/jcma.0000000000001225
- Citations
- 1
- Primary Topic
- Artificial Intelligence in Healthcare and Education
- Type
- article
- Field-Weighted Citation Impact
- 0.65