L2 Writers’ Generative AI Use Boosts Writing Quality, but not Uniformly: Cases of No Improvement

Although artificial intelligence (AI), particularly generative AI (GenAI), has been widely discussed in second language writing research as a promising form of support, earlier studies have not compared the quality of the same writers’ work when they compose with and without it. Furthermore, it remains uncertain whether GenAI consistently enhances writing quality across individuals. The present study employs both quantitative and qualitative approaches to assess whether GenAI improves the analytic quality of individual second language writers’ essays and to explore whether, and if so, why certain writers’ experience declines in particular components of writing quality. All participants completed two essays in a fixed sequence, first without GenAI assistance and then with it, which may constitute a potential limitation of the study design. The quantitative analyses indicated that, overall, essays written with GenAI assistance received higher scores across all four analytic components (task response, coherence and cohesion, lexical resource, and grammatical range and accuracy) than essays written without GenAI assistance. Yet this pattern was not consistent; some writers actually declined on specific components. The qualitative analyses then identified three themes that explain lower scores in particular domains when GenAI was used: (1) the prioritization of content and evidence during GenAI interaction over the management of the overall demands of the task, which led to inadequate task fulfillment and formatting; (2) the inclusion of GenAI suggestions without sufficient attention to integration or coherence; and (3) a relative lack of attention to vocabulary and grammar revision. These results point to the importance of instructional interventions that support effective GenAI use and AI literacy among second language writers, and suggest that GenAI-based systems should be customized to align with writers’ specific needs and the specific components of task requirements.

Authors

Institutions

Publication Details

Journal
RELC Journal
Published
2026-11-19
DOI
https://doi.org/10.1177/00336882261504063
Primary Topic
EFL/ESL Teaching and Learning
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

L2 Writers’ Generative AI Use Boosts Writing Quality, but not Uniformly: Cases of No Improvement

Haerim Hwang, Dian Jiang, Xingyu Chang
RELC Journal
EFL/ESL Teaching and Learning
article

L2 Writers’ Generative AI Use Boosts Writing Quality, but not Uniformly: Cases of No Improvement

Haerim Hwang, Dian Jiang, Xingyu Chang
article en

Abstract

Although artificial intelligence (AI), particularly generative AI (GenAI), has been widely discussed in second language writing research as a promising form of support, earlier studies have not compared the quality of the same writers’ work when they compose with and without it. Furthermore, it remains uncertain whether GenAI consistently enhances writing quality across individuals. The present study employs both quantitative and qualitative approaches to assess whether GenAI improves the analytic quality of individual second language writers’ essays and to explore whether, and if so, why certain writers’ experience declines in particular components of writing quality. All participants completed two essays in a fixed sequence, first without GenAI assistance and then with it, which may constitute a potential limitation of the study design. The quantitative analyses indicated that, overall, essays written with GenAI assistance received higher scores across all four analytic components (task response, coherence and cohesion, lexical resource, and grammatical range and accuracy) than essays written without GenAI assistance. Yet this pattern was not consistent; some writers actually declined on specific components. The qualitative analyses then identified three themes that explain lower scores in particular domains when GenAI was used: (1) the prioritization of content and evidence during GenAI interaction over the management of the overall demands of the task, which led to inadequate task fulfillment and formatting; (2) the inclusion of GenAI suggestions without sufficient attention to integration or coherence; and (3) a relative lack of attention to vocabulary and grammar revision. These results point to the importance of instructional interventions that support effective GenAI use and AI literacy among second language writers, and suggest that GenAI-based systems should be customized to align with writers’ specific needs and the specific components of task requirements.

RELC Journal
Chinese University of Hong Kong (HK)
EFL/ESL Teaching and Learning
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon