Design, methods, and theories in research on writing competence development in the GenAI writing era

This study presents a systematic review of the designs, methods, and theories used in empirical studies on AIpowered digital writing tools (AI-DWTs) within the context of Technology-Mediated Language Learning (TMLL).Despite the numerous systematic reviews on AI in education (AIED), reviews focusing on the designs, methods, and theoretical approaches used in investigating AI-DWTs in developing writing competence remain underexplored.Hence, this study addresses three research questions: (a) What research designs are used in studies on AI-DWTs for teaching and learning writing?(b) What research methodologies, including data collection and mode of analysis, are used in studies on AI-DWTs for teaching and learning writing? and (c) What theories/ models are most commonly employed in research on AI-DWTs for teaching and learning writing?The study found several experimental designs, mostly utilizing quantitative and mixed methods.Pre-tests and post-tests through writing tasks and questionnaires for assessing perceptions were prevalent data collection techniques.Thematic analysis was employed for data from survey and systematic review designs, while paired sample t-tests and independent-sample t-tests were used for analyzing test scores.However, most studies that implicitly leaned toward TMLL often lacked a solid theoretical foundation.This perennial challenge has implications for future research on AI-DWTs if not addressed.For future studies on AI-DWTs in enhancing writing competence, it is recommended to focus on direct observation of learner-AI-DWT interactions (LAII) as well.This would examine how learners engage with these tools, prompt them, employ strategies, process information, and how these interactions influence the recursive stages of the writing process.