Finding the fingerprints of generative AI in psychology publications

Generative artificial intelligence (AI) and in particular, large language models (LLMs) have become embedded in the research process, with scientists increasingly using LLMs to assist with manuscript drafting. Previous studies have documented the linguistic fingerprints of LLMs in biomedical and technical sciences, whereas psychology has remained largely unexplored. This study investigates the presence of AI-associated linguistic markers in psychology publications between 2020 and 2024. We constructed a corpus of 368,531 abstracts indexed in Scopus across 1,121 psychology journals and analysed the relative frequencies of 28 words previously identified as overrepresented in AI-assisted writing. Results indicated that the use of these stylistic markers remained stable from 2020 to 2022 but rose significantly following the release of ChatGPT, with relative frequencies increasing in 2023 and showing a substantial surge in 2024. Counterfactual modelling suggested that at least 9% of 2024 abstracts contained linguistic traces consistent with LLM editing. Group-level analyses revealed that increases were not confined to lower-prestige outlets as higher-ranked journals, major publishers, and both English- and non-English speaking countries all showed significant growth in AI-associated word use. These findings provide the first large-scale evidence that generative AI has measurably shaped the psychology publication record. The results raise critical implications for research integrity, transparency, and equity in scientific publishing. As psychology continues to address its credibility challenges, proactive standards, ethical guidelines, and detection practices will be essential for integrating AI tools responsibly into the scholarly ecosystem.

Authors

Publication Details

Published
2025-09-23
DOI
https://doi.org/10.31234/osf.io/3hx76_v1
Citations
1
Primary Topic
Artificial Intelligence in Healthcare and Education
Type
article
Field-Weighted Citation Impact
0.65
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Finding the fingerprints of generative AI in psychology publications

Ziwen Teuber, Jean‐Marc Dewaele, Elouise Botes, Nora Vitali et al.
1 citations
Artificial Intelligence in Healthcare and Education
0.65
article

Finding the fingerprints of generative AI in psychology publications

Ziwen Teuber, Jean‐Marc Dewaele, Elouise Botes, Nora Vitali, Joanne Colling
article en
1 citations

Abstract

Generative artificial intelligence (AI) and in particular, large language models (LLMs) have become embedded in the research process, with scientists increasingly using LLMs to assist with manuscript drafting. Previous studies have documented the linguistic fingerprints of LLMs in biomedical and technical sciences, whereas psychology has remained largely unexplored. This study investigates the presence of AI-associated linguistic markers in psychology publications between 2020 and 2024. We constructed a corpus of 368,531 abstracts indexed in Scopus across 1,121 psychology journals and analysed the relative frequencies of 28 words previously identified as overrepresented in AI-assisted writing. Results indicated that the use of these stylistic markers remained stable from 2020 to 2022 but rose significantly following the release of ChatGPT, with relative frequencies increasing in 2023 and showing a substantial surge in 2024. Counterfactual modelling suggested that at least 9% of 2024 abstracts contained linguistic traces consistent with LLM editing. Group-level analyses revealed that increases were not confined to lower-prestige outlets as higher-ranked journals, major publishers, and both English- and non-English speaking countries all showed significant growth in AI-associated word use. These findings provide the first large-scale evidence that generative AI has measurably shaped the psychology publication record. The results raise critical implications for research integrity, transparency, and equity in scientific publishing. As psychology continues to address its credibility challenges, proactive standards, ethical guidelines, and detection practices will be essential for integrating AI tools responsibly into the scholarly ecosystem.

Openalex Percentile: Top 21%
Artificial Intelligence in Healthcare and Education
0.65
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Finding the fingerprints of generative AI in psychology publications — Ziwen Teuber, Jean‐Marc Dewaele, et al. · (2025) | TGRS Research Map | TGRS