Exploring AI-generated feedback in peer-discussion contexts: A mixed-methods study of essay writing in secondary classrooms

This mixed-methods study investigates how an AI-powered writing tool providing automated feedback compares with peer-generated feedback in supporting secondary students' essay writing. Two research questions guided the study: (1) Did using an AI-powered tool with AI-generated feedback yield greater gains in writing quality from the first to the final draft compared with a standard editor with peer feedback? (2) How did students’ engagement in the writing process differ in the target (used AI-generated feedback) and comparison (used peer feedback) groups, as evidenced by teacher–student and peer–peer interactions and how were these patterns associated with their conceptual understanding of essay content? Eighty-one ninth-grade students from six Norwegian classrooms participated, with three classes using the AI-powered Essay Assessment Technology (EAT) and three relying on peer feedback. Quantitative analyses of first and final drafts showed that both groups improved, but students using EAT achieved statistically significant gains in writing quality. However, moderate inter-rater reliability limits the strength of these findings. Qualitative analysis of classroom video data revealed distinct engagement patterns. Students in the EAT group drew on AI-generated “covered” subthemes (ideas already present in their writing) and “suggested” subthemes (relevant ideas not yet included) to refine their essays, fostering more systematic discussions of essay content. In contrast, students in the peer-feedback group focused more on surface-level issues, such as spelling and word count, with less consistent attention to essay content. These findings suggest that AI-generated feedback, when embedded in peer discussion and teacher-facilitated classrooms, can strengthen the development of students’ conceptual understanding of essay content. Analyses indicate that structured AI feedback supported greater gains compared with peer feedback alone. The study highlights the pedagogical potential of AI-powered tools as part of formative assessment practices, while underscoring the critical role of teacher facilitation and structured feedback in fostering deeper engagement with essay content. • AI-supported feedback strengthened students' conceptual understanding of essay content. • EAT structured guidance through covered and suggested subthemes. • AI group achieved significant gains in writing quality over comparison group. • Galperin's phases applied to analyse learning trajectories. • AI feedback complemented teacher facilitation and scaffolded writing.