Performance of ChatGPT-4o on the Japanese Medical Licensing Examination: Evalution of Accuracy in Text-Only and Image-Based Questions
This study evaluated the performance of ChatGPT with GPT-4 Omni (GPT-4o) on the 118th Japanese Medical Licensing Examination. The study focused on both text-only and image-based questions. The model demonstrated a high level of accuracy overall, with no significant difference in performance between text-only and image-based questions. Common errors included clinical judgment mistakes and prioritization issues, underscoring the need for further improvement in the integration of artificial intelligence into medical education and practice.
Authors
- Yuki Miyazaki (ORCID: https://orcid.org/0000-0003-4952-0886)
- Masahiro Hata (ORCID: https://orcid.org/0000-0003-3837-1504)
- Manabu Ikeda (ORCID: https://orcid.org/0000-0003-0730-5774)
- Shun Takahashi (ORCID: https://orcid.org/0000-0002-9049-4694)
- Atsuya Hirashima
- Yuta Nakagawa
- Mitsuhiro Eto
- Hisaki Omori
Institutions
- Wakayama Medical University (JP)
- Kohnan Hospital (JP)
- Psychiatric Medical Center (AM)
- Osaka Metropolitan University (JP)
- Tokyo Metropolitan University (JP)
- The University of Osaka (JP)
Publication Details
- Journal
- JMIR Medical Education
- Published
- 2024-11-23
- DOI
- https://doi.org/10.2196/63129
- Citations
- 9
- Primary Topic
- Artificial Intelligence in Healthcare and Education
- Type
- article
- Field-Weighted Citation Impact
- 1.08