Optimizing GAI-assisted formative feedback: an experimental study on its effects on engagement, shared metacognition, and learning performance in online collaborative learning

Generative artificial intelligence (GAI) has the potential to assist instructors by providing effective formative feedback to students. Focus and complexity are important characteristics of formative feedback. Few studies have examined the combined effects of these factors in online collaborative learning. This study explored the effects of focus and complexity of GAI-assisted formative feedback (GAI-FF) on students’ online collaborative engagement, shared metacognition, And learning performance. A total of 114 preservice teachers were recruited from a public university in northwest China and randomized into four experimental conditions: task level and simple, task level and elaborated, multilevel and simple, and multilevel and elaborated GAI-FF. Participants were provided with four types of GAI-FFs longitudinally, and their pre-and post-tests, discussion processes, and learning performance were measured and recorded. Analysis of Variance (ANOVA) and its nonparametric alternatives were performed, and we found that (1) multi-level GAI-FF significantly promoted objective behavioral engagement, but not cognitive, positive, or negative emotional engagement; (2) the focus, not the complexity of GAI-FF, significantly impacted group shared metacognition; (3) there was a significant interaction effect between focus and complexity of GAI-FF on group shared metacognition; and (4) preservice teachers in multi-level GAI-FF conditions significantly outperformed those in task-level conditions in terms of group artifacts and course performance. The theoretical and practical implications of optimizing GAI-FF in collaborative learning to support learning and performance are discussed.