How do teachers process technology-based formative assessment results in their daily practice? Results from process mining of think-aloud data

Technology-based formative assessments are considered promising in terms of reducing teachers’ workload and providing validity advantages but little is known how teachers use the assessment results to inform their instruction in their daily practice. We explored how teachers process technology-based formative assessment results using think-aloud methodology in an ecologically valid setting. Forty-eight experienced in-service teachers participated in the study. We asked the teachers to verbalize their thoughts while they processed students’ formative assessment results as they usually do. Screencasts of the verbalizations and assessment results were recorded. Based on these, trained raters coded the main steps of processing and which specific aspects of the results were noticed based on a deductive-inductive coding scheme. Cluster analyses were applied to explore differences among teachers, and process mining was conducted to explore the main processes. We found four main steps: noticing results , comparing with personal perspective , analyzing errors and constructing instructional implications . Relative durations of these steps vary substantially among teachers. Cluster analyses indicate that processes were differentiated according to the complexity of summarizing and building relationships between single data points. The fitted process model revealed low dependency values in general and indicates that noticing results on its own seemed to be insufficient for constructing instructional implications. This study generates the hypothesis that analyzing errors and comparing results with the personal perspective are important for teachers for next instructional decisions. • We explored how teachers process formative assessments using think-aloud methodology. • Cluster analyses reveal groups that differ in the complexity of noticed aspects. • Process mining shows low dependency values in general. • Noticing results on its own seems insufficient for instructional implications.