Investigating the Effect of Visualization Literacy and Guidance on Teachers' Dashboard Interpretation

Recent research on learning analytics dashboards has focused on designing user interfaces that offer various forms of visualization guidance (often referring to notions such as data storytelling or narrative visualization) to teachers (e.g., emphasizing data points or trends with colour and adding annotations), aiding them in interpreting visual elements to gain a comprehensive understanding of students’ learning processes. Yet, while some studies have explored how teachers interpret students’ data through these dashboards, many have overlooked the diverse technical capabilities of teachers, which can significantly impact their use of LA dashboards. In particular, visualization literacy (VL) skills can greatly influence how effectively teachers interpret dashboards. To the best of our knowledge, no comprehensive account exists that details how teachers with varying VL skills interpret visual representations of students’ data. In this paper, we address this gap by investigating how teachers interpret LA dashboards, both with and without visualization guidance, taking into account their VL. We illustrate this by analyzing teachers’ think-aloud sessions as they engage with dashboards in the context of monitoring synchronous online learning tasks undertaken by student groups using Zoom and Google Docs. Using epistemic network analysis, we examine the differences in interpretations between teachers with varying VL levels. Our findings revealed that teachers with low VL exhibited shallower dashboard interpretations than those with high VL. However, the association of VL with successful task completion rate was not significant. Also, visualization guidance did not enable teachers to deepen their interpretations. While some visualization guidance helped teachers to complete tasks correctly, excessive visualization guidance can also be detrimental.