Leveraging Artificial Intelligence (AI) to Enhance Computer Science Instruction

Measuring and understanding instructional processes in classrooms is essential for enhancing teaching and learning outcomes. Classroom observations provide educators with integral opportunity for professional development, fostering reflective practice and continuous improvement. However, traditional observation and coaching procedures are time-consuming, cumbersome, and expensive, requiring travel or video coding. Artificial intelligence (AI) technologies present a promising solution, enabling streamlined, efficient, and nuanced analysis of classroom observation data. This study aims to develop AI-based computer vision and multi-modal models to measure computer science (CS) instructional processes in video data and validate their accuracy. Using two K-8 CS classroom instruction videos, we tested two deep-learning approaches: 1) Convolutional Neural Network (CNN) for analyzing visual imagery, and 2) a multimodal deep learning approach that analyzed both visual and auditory information. Results showed that the multimodal approach achieved the highest accuracy. Our findings support the broader application of these methods for analyzing more intricate teaching and learning processes. We discuss the implications of the results for CS education practice and research.