Enhancing Student Success through Self-Regulated Learning with LLMs

Generative Artificial Intelligence (GenAI) has, since the release of ChatGPT, sparked many debates regarding its role in education. The ability to generate human-like texts and pass programming exams creates concerns in how students in higher education retain long-lasting knowledge and skills to self-regulate their learning. While ChatGPT offers broad expertise, it is not specifically designed with educational objectives or curriculum in mind. This has led to a growing interest in purpose-built AI tools that support students’ learning processes. This exploratory study evaluates the impact of a Virtual Teaching Assistant (VTA) compared to ChatGPT on students’ cognitive retention and self-regulated learning (SRL) in higher education. The aim is to analyse whether a custom-built AI tutor fosters improved cognitive retention and self-regulated learning skills than unrestricted ChatGPT usage. 22 university students, who studied an introductory C programming course, participated in this study. By pre- and post-knowledge tests, including a Motivated Strategies for Learning Questionnaire, the participants' knowledge retention and SRL strategies were evaluated. Group A were assigned to utilize the standard ChatGPT-4 as their study tutor, whilst Group B utilized the VTA. Unlike ChatGPT, the VTA was designed to provide guided answers and assist the students rather than immediately give away all the solutions. The study shows that the use of the VTA did not lead to a statistically significant improvement in cognitive retention or SRL when compared to ChatGPT. However, the results of the exploratory correlation analysis indicate that specific SRL aspects, particularly ‘Critical Thinking’ in Group A and ‘Time and Study Environment’ in Group B, were positively associated with improved cognitive retention. These results suggest that tailored AI tools may foster different SRL strategies that impact learning outcomes. Further research is imperative to explore the long-term effects of AI-guided tools.