Performance of State-of-the-Art Multimodal Large Language Models on an Image-Rich Radiology Board Examination: Comparison to Human Examinees
Modern multimodal large language models, notably Gemini 2.5 Pro Preview and o3, surpassed average human performance on the 2024 Japanese Radiology Board Examination. Gemini models showed significant score improvements when utilizing image data and offer top-tier performance at a competitive cost, indicating rapid advancements and excellent cost-effectiveness for radiology applications.
Authors
- Takanori Masuda (ORCID: https://orcid.org/0000-0002-4164-4369)
- Masafumi Kidoh (ORCID: https://orcid.org/0000-0002-9589-7054)
- Toshinori Hirai (ORCID: https://orcid.org/0000-0003-1480-2197)
- Yoshinori Funama (ORCID: https://orcid.org/0000-0003-1797-9625)
- Hiroyuki Uetani (ORCID: https://orcid.org/0000-0002-0547-1210)
- Naoki Kobayashi (ORCID: https://orcid.org/0009-0006-0866-8517)
- Takeshi Nakaura (ORCID: https://orcid.org/0000-0002-9010-0341)
- Yasunori Nagayama (ORCID: https://orcid.org/0000-0003-3280-7346)
- Seitaro Oda (ORCID: https://orcid.org/0000-0002-0609-2191)
Institutions
- Kumamoto University Hospital (JP)
- Kawasaki University of Medical Welfare (JP)
- Kumamoto University (JP)
Publication Details
- Journal
- Academic Radiology
- Published
- 2025-11-29
- DOI
- https://doi.org/10.1016/j.acra.2025.10.051
- Citations
- 1
- Primary Topic
- Artificial Intelligence in Healthcare and Education
- Type
- article
- Field-Weighted Citation Impact
- 0.65