Assessing accuracy and legitimacy of multimodal large language models on Japan Diagnostic Radiology Board Examination
Recent multimodal LLMs, particularly o3 and Gemini 2.5 Pro, have demonstrated remarkable progress on JDRBE questions, reflecting their rapid evolution in diagnostic radiology. Eight multimodal large language models were evaluated on the Japan Diagnostic Radiology Board Examination. OpenAI's o3 and Google DeepMind's Gemini 2.5 Pro achieved high accuracy rates (72% and 70%) and received good legitimacy scores from human raters, demonstrating steady progress.
Authors
- Yosuke Yamagishi (ORCID: https://orcid.org/0009-0006-7688-3075)
- Tomohiro Kikuchi (ORCID: https://orcid.org/0000-0002-4222-4569)
- Osamu Abe (ORCID: https://orcid.org/0000-0002-1180-2629)
- Soichiro Miki (ORCID: https://orcid.org/0000-0001-6365-6531)
- Yuichiro Hirano (ORCID: https://orcid.org/0000-0003-1564-0513)
- T. Yoshikawa (ORCID: https://orcid.org/0000-0002-1924-5468)
- Takahiro Nakao (ORCID: https://orcid.org/0000-0001-9498-7501)
- Shouhei Hanaoka (ORCID: https://orcid.org/0000-0002-7496-1651)
- Yukihiro Nomura (ORCID: https://orcid.org/0000-0001-6471-9936)
- Yuta Nakamura (ORCID: https://orcid.org/0000-0001-6962-6704)
Institutions
- Jichi Medical University (JP)
- Chiba University (JP)
- University of Tokyo Hospital (JP)
Publication Details
- Journal
- Japanese Journal of Radiology
- Published
- 2025-09-12
- DOI
- https://doi.org/10.1007/s11604-025-01861-y
- Citations
- 4
- Primary Topic
- Radiology practices and education
- Type
- article
- Field-Weighted Citation Impact
- 5.44