An Augmented Reality Framework for Education: Deep Learning Integration and Impact Evaluation

We propose a novel augmented reality (AR) framework that transforms education with immersive and interactive experiences that enhance student engagement and comprehension. However, existing AR-based learning solutions are built upon generic object detectors that struggle with recognizing domain-specific educational materials, thus limiting their effectiveness. To address this challenge, we introduce a novel fully automated image dataset generation pipeline that synthesizes high-fidelity images from 3D models by varying lighting, camera angles, and background occlusion. This variation enhances the diversity of the dataset, enabling robust training of domain-specific object detectors. The proposed pipeline stands out from existing methods because it provides scalability and adaptability features that allow researchers to build customized educational datasets. The study involved generating a dataset and evaluating it using four state-of-the-art object detection models: Faster R-CNN, SSD, YOLOv5n, and YOLOv7. The YOLOv7 detection model reached an accuracy of 97.2% with 99.5% [email protected] and performed at a real-time speed of 45 frames per second (FPS) making it the best choice for AR applications. To assess the educational impact of our system, we conducted a pilot study involving 210 elementary students. The results showed notable improvements in learning outcomes: fourth graders’ scores increased from <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$68.78~\pm ~10.85$ </tex-math></inline-formula> to <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$90.96~\pm ~11.70$ </tex-math></inline-formula>, while fifth graders improved from <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$62.43~\pm ~11.53$ </tex-math></inline-formula> to <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$75.43~\pm ~12.53$ </tex-math></inline-formula>. Comprehensive statistical analyses, including ANOVA, regression, and paired t-tests, confirmed that our approach significantly enhances both academic performance and student engagement when compared to traditional learning methods. These findings demonstrate that our domain-focused data pipeline and optimized object detection framework effectively bridge the gap between deep learning research and AR real-world classroom implementation, offering a highly scalable and transformative solution for AR-based education.