AI- vs Human-Based Assessment of Medical Interview Transcripts in a Generative AI–Simulated Patient System: Cross-Sectional Validation Study
ABA-o1 and ABA-5 produced scores closely matching HBA while demonstrating superior consistency and reliability. In the setting of virtual interview transcripts, these findings suggest that ABA may serve as a valid, rapid, and scalable alternative to HBA, reducing per-assessment time by over half. Applied strategically, AI-based scoring could enable timely feedback, improve efficiency, and reduce faculty workload. Further research is needed to confirm generalizability across broader settings.
Authors
- Minoru Kishi (ORCID: https://orcid.org/0009-0001-5585-4519)
- Yuji Yamada (ORCID: https://orcid.org/0000-0002-5452-5334)
- Hiromizu Takahashi (ORCID: https://orcid.org/0000-0002-5055-7937)
- Takeshi Kondo (ORCID: https://orcid.org/0000-0002-3307-671X)
- Tadashi Matsuura (ORCID: https://orcid.org/0009-0007-1767-6540)
- Ayame Enomoto (ORCID: https://orcid.org/0009-0006-9022-2126)
- Fumiaki Sano (ORCID: https://orcid.org/0009-0001-8880-8367)
- Toshio Naito (ORCID: https://orcid.org/0000-0003-1646-9930)
- Yoshinori Tokushima (ORCID: https://orcid.org/0009-0008-5473-9798)
- Sho Nagai (ORCID: https://orcid.org/0009-0001-2363-1177)
- Kiyoshi Shikino (ORCID: https://orcid.org/0000-0002-3721-3443)
- Tsutomu Shinohara (ORCID: https://orcid.org/0000-0002-1562-2043)
- Yoshitaka Tomoda (ORCID: https://orcid.org/0000-0003-0115-7379)
- Roselaine Terezinha Migotto Watanabe (ORCID: https://orcid.org/0009-0007-5239-6720)
- Yuki Aiyama (ORCID: https://orcid.org/0009-0009-9608-1135)
Institutions
- Tokyo Medical and Dental University (JP)
- Chiba University (JP)
- Saga University (JP)
- Tenri Hospital (JP)
- Juntendo University (JP)
- Maastricht University (NL)
- Nagoya University Hospital (JP)
- Tokyo-Kita Medical Center (JP)
- Hokkaido hospital (JP)
- Mie Chuo Medical Center (JP)
- Kashiwa Municipal Hospital (JP)
- University of Human Environments (JP)
- Icahn School of Medicine at Mount Sinai (US)
Publication Details
- Journal
- JMIR Medical Education
- Published
- 2025-12-15
- DOI
- https://doi.org/10.2196/81673
- Primary Topic
- Artificial Intelligence in Healthcare and Education
- Type
- article
- Field-Weighted Citation Impact
- 0.00