Performance of 5 AI Models on United States Medical Licensing Examination Step 1 Questions: Comparative Observational Study
AI models showed varying strengths across domains, with Grok demonstrating the highest accuracy and consistency in this dataset, particularly for image-based and reasoning-heavy questions. Although ChatGPT-4 remains widely used, newer models like Grok and Copilot also performed competitively. Continuous evaluation is essential as AI tools rapidly evolve.
Authors
- Dania El Natour (ORCID: https://orcid.org/0009-0007-1046-7228)
- Reda Assi
- Mohamad Abou Alfa
- Toufic Dally
- Bahaa Bou Dargham
- Ahmad Chaaban
Institutions
- Beirut Arab University (LB)
- Hammoud Hospital University Medical Center (LB)
Publication Details
- Journal
- JMIR AI
- Published
- 2026-02-09
- DOI
- https://doi.org/10.2196/76928
- Primary Topic
- Artificial Intelligence in Healthcare and Education
- Type
- article
- Field-Weighted Citation Impact
- 0.00