‘Hey, I See You. Do You Need Help?’: Understanding the Algorithmic Gaze of AI-Driven Student Surveillance

Abstract The adoption of Artificial Intelligence (AI)-driven surveillance systems in educational settings is becoming increasingly common as policymakers, school leaders, and educators seek to enhance student safety and wellbeing. Understanding what student data are collected, how monitoring occurs, and what assumptions are embedded within these systems is therefore essential to informing ethical approaches to educational technology governance. This exploratory paper investigates the potential implications of AI-driven surveillance in Australian K-12 schools. Using a qualitative case study design, it examines three surveillance tools through the conceptual lens of the algorithmic gaze. The impacts of the algorithmic gaze are particularly concerning in educational settings where AI technologies are increasingly used to monitor students’ behaviour and mental health, often without students fully understanding how their actions, opportunities, and experiences may be mediated by code. To reveal how this process operates, the study undertakes a thematic analysis of product website discourses, including product marketing pages, customer story blogs, and support documentation. More specifically, the paper examines the learning, psychological, and equity harms that might arise from this form of surveillance in schools.