A meta-synthesis of automatic writing evaluation research: trends and developments over a decade

Abstract Automated writing evaluation (AWE) technologies have emerged as promising tools that streamline the feedback process and strengthen students’ writing skills. This meta-review synthesized eleven systematic reviews and meta-analyses on AWE research published from 2015 to 2025. Before the main analyses, all selected reviews were evaluated using Many-Facet Rasch Model (MFRM) to determine the study quality. Next, syntheses methods employed narrative approach and text mining analysis. The results suggested the shift from rule-based AWE system to AI-driven AWE tools over three decades. The synthesized findings from meta-analyses supported the effectiveness of AWE on surface-level writing (e.g., grammar, spelling) but highlighted its limitations in improving high-order level of writing (e.g., argumentation). Further, drawing on moderator analyses, educational levels and duration deserve attention in the implementation of AWE. Finally, persistent challenges, future research directions, and practical pedagogy were also identified and discussed. Overall, the present meta-synthesis study supports the potential value of AWE as an adjunct tool rather than a replacement for human feedback in writing instruction.