Effectiveness and students’ perception of tutor-guided AI navigation in undergraduate medical education in Sri Lanka: a quasi-experimental study

Abstract Background The integration of Artificial Intelligence (AI), particularly Large Language Models (LLMs) such as ChatGPT, into undergraduate medical education offers new opportunities for personalized learning. However, concerns remain regarding content accuracy, possible over-reliance and limited critical engagement when AI tools are used without guidance. Tutor-guided AI navigation has been proposed as a structured approach combining AI accessibility with educator oversight. This study aimed to evaluate the effectiveness and perception of tutor-guided AI navigation among medical undergraduates. Methods The study was conducted among 87 final-year medical students at the Faculty of Medicine, University of Kelaniya, Sri Lanka. A quasi-experimental pre-test/post-test design compared knowledge gains following tutor-guided versus self-directed ChatGPT-assisted learning. A cross-sectional survey assessing three domains-Perception of Learning Experience, AI Usability and Confidence, Satisfaction and Overall Impression- was administered immediately after the session. Responses were measured using a five-point Likert scale. Internal consistency was evaluated with Cronbach’s alpha and descriptive statistics were analysed using SPSS. Results Baseline pre-test scores were comparable between groups (W = 670.5, p = 0.153). Both groups improved significantly from pre- to post-test ( p < 0.001), with median scores increasing from 4 to 7. Knowledge gain between groups was not statistically significant (Welch’s t = 1.07, p = 0.144), although the effect size (Hedges’ g = 0.36) suggested a small-to-moderate advantage for tutor guidance. In the perception survey, reliability was high (Cronbach’s alpha > 0.86). Students rated ease of use (4.19 ± 1.07), tutor-guided AI experience (4.03 ± 1.13), satisfaction (3.91 ± 0.94) and willingness to recommend (4.01 ± 0.97) favourably, while trust in AI accuracy was moderate (3.27 ± 0.91). Conclusion Tutor-guided AI-assisted learning was well accepted by students and provided a positive, structured learning experience. Although it did not produce a statistically significant advantage in immediate knowledge gain over self-directed AI use, it was associated with favourable perceptions of usability, satisfaction and recommendation. These findings support the structured integration of tutor-guided AI into undergraduate medical education while maintaining educator oversight and critical appraisal of AI-generated content.