Style, sentiment, and quality of undergraduate writing in the AI era: A cross-sectional and longitudinal analysis of 4,820 authentic empirical reports

As generative artificial intelligence (GenAI) becomes widespread in education, its influence on students’ academic practice raises concern. We conducted pre-registered analyses of 4,820 empirical reports authored by 2,000 psychology undergraduates (2016-2025) to examine how ChatGPT’s launch in November 2022 may influence the style, sentiment, and quality of students’ writing. Following ChatGPT’s release, prevalence of ChatGPT-associated lexical markers (e.g., delve, intricate ) surged until 2024, then declined in 2025, possibly due to some students actively masking GenAI traces. Writing style (indexed by lexical diversity/density/sophistication, nominalisation, readability) became increasingly formal post-ChatGPT, diverging from predicted/historical trends. Consistent with GenAI’s positivity bias, sentiment also became more positive, independent of a report’s topic or the statistical significance of the reported results, raising questions about its potential impact on students’ voice, creativity, and critical thinking. Despite the stylistic changes, writing quality (indexed by grades and feedback) showed no discernible shifts. In an exploratory analysis, we asked GPT models to rewrite pre-ChatGPT reports, and these GPT-rewritten texts resembled post-ChatGPT reports more in style and sentiment, providing evidence for GPT models driving the observed shifts. Finally, no students disclosed AI use despite clear guidelines, highlighting the ineffectiveness of voluntary disclosure and the need for more enforceable policy. Overall, our findings indicate that a growing proportion of undergraduate students are incorporating GenAI in their written assignments, producing work that exhibits stylistic and tonal features characteristic of ChatGPT-generated text. • We analysed 4820 psychology empirical reports from 2016 to 2025 • We found clear shifts in style and sentiment following ChatGPT’s release • Lexical markers typical of GPT-generated text jumped in 2024 but declined in 2025 • After ChatGPT’s launch, students used more formal, positive language • These changes occurred without corresponding improvements in grades or feedback • Student submissions are becoming increasingly similar to GPT-generated texts

Authors

Institutions

Publication Details

Journal
Computers and Education Artificial Intelligence
Published
2025-11-26
DOI
https://doi.org/10.1016/j.caeai.2025.100507
Primary Topic
Artificial Intelligence in Healthcare and Education
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Style, sentiment, and quality of undergraduate writing in the AI era: A cross-sectional and longitudinal analysis of 4,820 authentic empirical reports

Matthew HC Mak, Lukasz Walasek
Computers and Education Artificial Intelligence
Artificial Intelligence in Healthcare and Education
article

Style, sentiment, and quality of undergraduate writing in the AI era: A cross-sectional and longitudinal analysis of 4,820 authentic empirical reports

Matthew HC Mak, Lukasz Walasek
article en

Abstract

As generative artificial intelligence (GenAI) becomes widespread in education, its influence on students’ academic practice raises concern. We conducted pre-registered analyses of 4,820 empirical reports authored by 2,000 psychology undergraduates (2016-2025) to examine how ChatGPT’s launch in November 2022 may influence the style, sentiment, and quality of students’ writing. Following ChatGPT’s release, prevalence of ChatGPT-associated lexical markers (e.g., delve, intricate ) surged until 2024, then declined in 2025, possibly due to some students actively masking GenAI traces. Writing style (indexed by lexical diversity/density/sophistication, nominalisation, readability) became increasingly formal post-ChatGPT, diverging from predicted/historical trends. Consistent with GenAI’s positivity bias, sentiment also became more positive, independent of a report’s topic or the statistical significance of the reported results, raising questions about its potential impact on students’ voice, creativity, and critical thinking. Despite the stylistic changes, writing quality (indexed by grades and feedback) showed no discernible shifts. In an exploratory analysis, we asked GPT models to rewrite pre-ChatGPT reports, and these GPT-rewritten texts resembled post-ChatGPT reports more in style and sentiment, providing evidence for GPT models driving the observed shifts. Finally, no students disclosed AI use despite clear guidelines, highlighting the ineffectiveness of voluntary disclosure and the need for more enforceable policy. Overall, our findings indicate that a growing proportion of undergraduate students are incorporating GenAI in their written assignments, producing work that exhibits stylistic and tonal features characteristic of ChatGPT-generated text. • We analysed 4820 psychology empirical reports from 2016 to 2025 • We found clear shifts in style and sentiment following ChatGPT’s release • Lexical markers typical of GPT-generated text jumped in 2024 but declined in 2025 • After ChatGPT’s launch, students used more formal, positive language • These changes occurred without corresponding improvements in grades or feedback • Student submissions are becoming increasingly similar to GPT-generated texts

Computers and Education Artificial IntelligenceVol. 9
University of Warwick (GB)
Openalex Percentile: Top 70%
Artificial Intelligence in Healthcare and Education
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.