Evaluating the value of AI-generated questions for USMLE step 1 preparation: A study using ChatGPT-3.5
We demonstrate that ChatGPT-3.5 can successfully generate Step 1 style questions with reasonable factual accuracy, and this method may be used by medical students preparing for USMLE examinations. While AI-generated questions demonstrated adequate factual accuracy, targeted prompting techniques should be used to overcome ChatGPT's bias towards particular medical conditions.
Authors
- Daniel A. Donoho (ORCID: https://orcid.org/0000-0002-0531-1436)
- Alan Balu (ORCID: https://orcid.org/0000-0001-9974-906X)
- Stefan T. Prvulovic (ORCID: https://orcid.org/0009-0006-4955-7696)
- Alexander Kim (ORCID: https://orcid.org/0000-0003-3355-3122)
- Claudia Fernandez Perez
- Gregory Keating
Institutions
- Georgetown University (US)
- MedStar Georgetown University Hospital (US)
- National Hospital (IN)
Publication Details
- Journal
- Medical Teacher
- Published
- 2025-03-27
- DOI
- https://doi.org/10.1080/0142159x.2025.2478872
- Citations
- 5
- Primary Topic
- Artificial Intelligence in Healthcare and Education
- Type
- article
- Field-Weighted Citation Impact
- 3.27