Leveraging Large Language Models to Evaluate the Quality of Narrative Feedback for Surgery Residents in Competency-Based Medical Education
Fine-tuned GPT-3.5 demonstrated strong potential for automating the evaluation of narrative feedback quality for surgery residents. However, LLM performance depends on the task and how well task structure aligns with the LLM architecture. LLM use in CBME may facilitate continuous quality improvement, providing faculty with automated feedback on their feedback.
Authors
- Neda Aghaei
- Boris Zevin (ORCID: https://orcid.org/0000-0003-2450-1865)
- Benjamin Y. M. Kwan (ORCID: https://orcid.org/0000-0002-1124-3206)
- Ingrid de Vries (ORCID: https://orcid.org/0009-0006-2529-2187)
- Zier Zhou (ORCID: https://orcid.org/0000-0002-4890-2842)
- Tessa Hanmore (ORCID: https://orcid.org/0000-0002-4654-9969)
- Nick Rogoza
Institutions
- Queen's University (CA)
- Logikos (United States) (US)
Publication Details
- Journal
- Annals of Surgery Open
- Published
- 2025-09-24
- DOI
- https://doi.org/10.1097/as9.0000000000000608
- Primary Topic
- Innovations in Medical Education
- Type
- article
- Field-Weighted Citation Impact
- 0.00