Style, sentiment, and quality of undergraduate writing in the AI era: A cross-sectional and longitudinal analysis of 4,820 authentic empirical reports
As generative artificial intelligence (GenAI) becomes widespread in education, its influence on students’ academic practice raises concern. We conducted pre-registered analyses of 4,820 empirical reports authored by 2,000 psychology undergraduates (2016-2025) to examine how ChatGPT’s launch in November 2022 may influence the style, sentiment, and quality of students’ writing. Following ChatGPT’s release, prevalence of ChatGPT-associated lexical markers (e.g., delve, intricate ) surged until 2024, then declined in 2025, possibly due to some students actively masking GenAI traces. Writing style (indexed by lexical diversity/density/sophistication, nominalisation, readability) became increasingly formal post-ChatGPT, diverging from predicted/historical trends. Consistent with GenAI’s positivity bias, sentiment also became more positive, independent of a report’s topic or the statistical significance of the reported results, raising questions about its potential impact on students’ voice, creativity, and critical thinking. Despite the stylistic changes, writing quality (indexed by grades and feedback) showed no discernible shifts. In an exploratory analysis, we asked GPT models to rewrite pre-ChatGPT reports, and these GPT-rewritten texts resembled post-ChatGPT reports more in style and sentiment, providing evidence for GPT models driving the observed shifts. Finally, no students disclosed AI use despite clear guidelines, highlighting the ineffectiveness of voluntary disclosure and the need for more enforceable policy. Overall, our findings indicate that a growing proportion of undergraduate students are incorporating GenAI in their written assignments, producing work that exhibits stylistic and tonal features characteristic of ChatGPT-generated text. • We analysed 4820 psychology empirical reports from 2016 to 2025 • We found clear shifts in style and sentiment following ChatGPT’s release • Lexical markers typical of GPT-generated text jumped in 2024 but declined in 2025 • After ChatGPT’s launch, students used more formal, positive language • These changes occurred without corresponding improvements in grades or feedback • Student submissions are becoming increasingly similar to GPT-generated texts
Authors
- Matthew HC Mak (ORCID: https://orcid.org/0000-0001-7237-4931)
- Lukasz Walasek (ORCID: https://orcid.org/0000-0002-7360-0037)
Institutions
- University of Warwick (GB)
Publication Details
- Journal
- Computers and Education Artificial Intelligence
- Published
- 2025-11-26
- DOI
- https://doi.org/10.1016/j.caeai.2025.100507
- Primary Topic
- Artificial Intelligence in Healthcare and Education
- Type
- article
- Field-Weighted Citation Impact
- 0.00