Exploring Student Engagement and Personalized Learning with a Retrieval-Augmented Generation (RAG) Chatbot

Students have become increasingly familiar with ChatGPT, a generative artificial intelligence chatbot that offers users quick access to information, engages them through dynamic question-and-answer interactions and provides feedback on tasks. However, a key challenge with ChatGPT in education is its propensity to hallucinate, which can affect learning outcomes. To address this, the TP AI Assistant, a Retrieval-Augmented Generation (RAG)-based chatbot, was developed to provide a safer and more structured online learning environment. Unlike ChatGPT, which generates broad and diverse responses, the TP AI Assistant focuses on delivering accurate and relevant information aligned with specific learning objectives. This ensures students receive content that aligns with their subject requirements. The TP AI Assistant was rolled out to approximately 600 Year 1 students and 600 Year 2 students from the same academic school at a public polytechnic in Singapore. Before triangulating with student perceptions, it is essential to establish a foundational understanding of their engagement patterns and learning behaviors through factual, unbiased logs. This study thus investigates how students interacted with this RAG-based chatbot and how its structured design supported self-regulated learning (SRL). Additionally, the study examines the strengths and limitations of using the RAG approach for personalizing student learning. Findings will provide insights into how a RAG-based chatbot can enhance learning experiences and better meet students' expectations.