Machine Learning-Based Classification of Programming Logic Understanding Levels by Mouse-Tracking Heatmaps

In the past decade, there has been a significant increase in the growth of the global technology employment market, primarily owing to the digital age, artificial intelligence expansion, cloud computing, and cybersecurity requirements. Consequently, the demand for IT professionals has increased. Introductory programming is a crucial early course in IT higher education as it aids in developing problem-solving skills. However, such courses tend to result in substantial failure and attrition rates. Some studies have proposed solutions such as peer learning and personalized feedback, but challenges persist. One key assumption is that teacher support is likely to help mitigate these rates. However, with limited class time, teachers find it challenging to support all the students. This study aims to automatically assess the programming logic understanding level (PLUL) during introductory programming courses. We proposed a method for classifying PLUL through a block-coding learning platform based on mouse-tracking heatmaps using machine learning techniques. We collected actual learning-task data from first-year university students and compared the performance of five machine learning models. The decision tree algorithm achieved an accuracy score of 76.00%, F1 score of 77.36%, sensitivity score of 75.93%, and specificity score of 76.09%. These findings demonstrate that our method can effectively classify students based on their PLUL of tasks on a block coding learning platform.