Beyond the “Wow” Factor: Using Generative AI for Increasing Generative Sense-Making

Abstract Generative artificial intelligence (GenAI) has emerged as a transformative tool in education, offering scalable individualized learning. However, there is a lack of theoretically informed and methodologically rigorous research on how GenAI can effectively augment learning. This manuscript addresses this gap by investigating the potential of a theory-informed GenAI chatbot, ChatTutor—designed for students—to facilitate generative sense-making by leveraging principles from generative learning theory. The study aimed to (1) extend theory by proposing how GenAI can support generative sense-making and (2) test its effects on conceptual knowledge, self-efficacy, and trust post-use and on conceptual knowledge, enjoyment, and behavioral intentions in a delayed follow-up. We conducted two between-subjects design experiments. Study 1 was a pre-registered experiment with 175 university students in an authentic cognitive psychology course, comparing ChatTutor with a generic GenAI system (ChatGPT) and a teaching-as-usual control. Study 2 replicated the design with 234 high school students, comparing ChatTutor to ChatGPT, and a re-study control. Results showed that ChatTutor significantly enhanced trust and enjoyment in both studies, and behavioral intentions in Study 1, compared to ChatGPT. It also improved conceptual knowledge over ChatGPT immediately in both studies and at follow-up in Study 2. ChatTutor outperformed the control condition in conceptual knowledge, although this difference was only significant in the follow-up of Study 1 and the immediate test of Study 2. Differences in self-efficacy were not significant. The research underscores the importance of integrating human-centered design and educational psychology theories into GenAI applications and offers directions for future research and practice.