Subject-Specialized Chatbot in Higher Education as a Tutor for Autonomous Exam Preparation: Analysis of the Impact on Academic Performance and Students’ Perception of Its Usefulness

This study evaluates the impact of an AI chatbot as a support tool for second-year students in the Bachelor’s Degree in Early Childhood Education program during final exam preparation. Over 1-month, 42 students used the chatbot, generating 704 interactions across 186 conversations. The study aimed to assess the chatbot’s effectiveness in resolving specific questions, enhancing concept comprehension, and preparing for exams. Methods included surveys, in-depth interviews, and analysis of chatbot interactions. Results showed that the chatbot was highly effective in clarifying doubts (91.4%) and aiding concept understanding (95.7%), although its perceived usefulness was lower in content review (42.9%) and exam simulations (45.4%). Students with moderate chatbot use achieved better academic outcomes, while excessive use did not lead to further improvements. The study also identified challenges in students’ ability to formulate effective questions, limiting the chatbot’s potential in some areas. Overall, the chatbot was valued for fostering study autonomy, though improvements are needed in features supporting motivation and study organization. These findings highlight the potential of chatbots as complementary learning tools but underscore the need for better user training in “prompt engineering” to maximize their effectiveness.