Using Artificial Intelligence for Eliciting Diagnostic Evidence From Students’ Drawings

Abstract: While artificial intelligence (AI) gained attention for eliciting diagnostic evidence from text answers using NLP or for generating visual stimuli, few studies investigate its use for analyzing visual data such as free-hand sketches from graphical response formats. The present case study is based on a formative assessment including instructional considerations and illustrates the application of three AI approaches to graphical responses from 96 students. Students answered two tasks assessing the conceptual understanding of fractions. Comparisons of AI approaches to expert ratings reveal promising results of two approaches (rule-based approach and ResNet). The third approach using a pretrained clip model showed lower performance, especially in tasks requiring counting. Additional comparisons to diagnostic evidence from other items highlight the relevance of graphical response items as a distinct item format. We discuss strengths and weaknesses of the approaches, as well as the case study, and hint to topics for further research.