Artificial intelligence teaching assistants: a scalable solution for supporting struggling medical students

PURPOSE: Large language models (LLMs), such as OpenAI's ChatGPT, have demonstrated tutoring benefits in small-scale pilot studies within focused areas of medical education. This study evaluated the large-scale implementation of AI-teaching assistants (AI-TAs) within a compulsory medical school course. METHOD: A quasi-experimental observational study with a mixed-methods was conducted to assess the impact of AI-TAs in a compulsory first-year medical school course at the University of Toronto in 2024. The research team developed AI-TAs using OpenAI's ChatGPT-4o and introduced them as a supplementary resource at the course's midpoint. They analyzed exam performance among the students who used AI-TAs (n = 87) and students who did not (n = 206). Additionally, surveys (n = 18) and interviews (n = 10) explored student perceptions of AI-TAs' effectiveness, usability, and impact on learning. RESULTS: Students who would later adopt AI-TAs had significantly lower pre-intervention exam scores than their peers (83.8% vs 88.1%; t(118.8)=-3.82, P<.001). They also more often failed to meet the course's assessment standard pre-intervention (24.1% vs 6.4%). After AI-TA adoption, performance converged for both early users (86.1% vs 86.5%; t(95.63)=-0.35, P=.72) and late adopters (85.6% vs 86.5%; t(26.08)=-0.52, P=.61), with similar proportions falling below the course's standard (4.4-6.4%). Thematic analysis and surveys identified three key advantages of AI-TAs: (1) Reliable and accurate educational support, (2) Efficient application across learning activities, and (3) Improved psychological safety, promoting engagement in active learning. CONCLUSIONS: AI-TAs correlated with improved exam performance, fewer students in academic difficulty, and enhanced student engagement through a psychologically safe learning environment. These findings suggest that AI-TAs can serve as a scalable, cost-effective tool to support struggling students, while complementing traditional instruction.