AR-Classroom: Integrating Conversational Artificial Intelligence with Augmented Reality Technology for Learning Spatial Transformations and Their Matrix Representation

This research full paper describes the AR-Classroom application that utilizes augmented reality (AR) and physical and virtual manipulatives to enable undergraduate students to build intuition about the relation between spatial transformations and their mathematical representations. To further build on the app's usability and functionality, additional features are being prototyped to continue improving the user-app interaction with the AR-Classroom. Some of the challenges the students faced when using AR-Classroom were recalling basic matrix operations without geometric context, basic trigonometric functions and their applications in the two-dimensional space, loss of AR registration for not understanding the AR environment, and User Interface (UI) issues. To address these issues, a conversational Artificial Intelligence (AI)-based multi-sensory and interactive assistance has been added to the AR-Classroom. Integrating sophisticated language processing and response generation of AI with immersive three-dimensional capabilities of AR can create a more engaging learning experience than the previous versions of the app. This integration focuses on creating a symbiosis between AR and AI. It creates an elevated user experience by offering real-time, personalized assistance to students dealing with issues related to understanding mathematical concepts and functionalities of the app. A qualitative exploratory usability study was done to assess the user's interaction with the AI implemented in the AR-Classroom, aiming to explore the AI's ability to guide students in using AR technology and aid in introductory matrix algebra learning, to effectively serve the students' learning. Based on the thematic analysis of the user experiment we found four main themes related to users' perceptions of AR-Classroom AI features usability: (1) AI chatbot ease-of-use, (2) Need for answer elaboration from AI, (3) Desire for visual information, and (4) Increased understanding of the content area. The scores of ease of use indicate AI's ability to guide complex tasks in an AR environment using AI features with less concern for the cognitive load. The overall result suggests the need for further investigation on incorporating AI-guided visual cues in an AR environment.