Extracting Learning Data From Handwritten Notes: A New Approach to Educational Data Analysis Based on Image Segmentation and Generative AI

While handwritten notes offer valuable insights into students’ knowledge retention, traditional analysis methods are often time-consuming and limited in scope. This study introduces an efficient approach for educational data analysis by combining image segmentation with generative AI to extract learning insights from students’ handwritten notes. Leveraging the Attention Multi-task U-Net for accurate segmentation and GPT-4o for content analysis, our method precisely identifies and categorizes text, charts, and formulas within notes. In the comparison experiment with SAM, the text segmentation accuracy is 5.46% higher than SAM, and the formula segmentation accuracy is 0.8% higher than SAM. The extracted data provides educators with a detailed view of students’ knowledge retention, highlights the areas students focus on, and identifies critical knowledge points that may be missing from notes. Our experiments on student notes from a Digital Signal Processing course demonstrate the method’s high accuracy and significant efficiency improvements in teachers’ review of student notes. This research contributes to educational technology and data mining by introducing an automated, scalable method that supports more personalized and effective educational strategies.