Are Scientists Starting to Sound like ChatGPT? Stylistic Diffusion in Biomedical Abstracts in the LLM Era

Large language models are part of everyday scholarly writing, but their influence may not always appear as obvious AI authorship. More often, it may appear through gradual changes in academic style. This study examines whether biomedical abstracts show measurable signs of LLM-associated “same-ification” after the public release of ChatGPT. Instead of classifying individual abstracts as AI-written, our study analyses population-level changes across 28,415 PubMed abstracts published between 2018 and 2025. A strict dictionary of LLM-associated stylistic markers was used alongside a control vocabulary, lexical diversity, semantic similarity, interrupted time-series modelling, journal-level heterogeneity analysis, and a study-specific validation framework called SAME-ML. The findings show a clear post-2022 stylistic shift. Mean strict marker use increased from 4.995 to 11.658 markers per 1000 words, representing a 133.4% rise, while the control vocabulary slightly declined. This increase was strongest in 2024 and 2025 and was mainly driven by markers linked to value framing, contribution signalling, connective phrasing, improvement rhetoric, and problem-solution positioning. SAME-ML further showed that the post-ChatGPT period contained a learnable stylistic signal that remained visible after topic-masked and matched-sample validation. Overall, the findings show a corpus-level rhetorical shift and provide a reproducible approach for tracking stylistic change in scholarly communication over time.

Authors

Institutions

Publication Details

Journal
Machine Learning and Knowledge Extraction
Published
2026-07-27
DOI
https://doi.org/10.3390/make8080223
Primary Topic
Artificial Intelligence in Healthcare and Education
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
article

Are Scientists Starting to Sound like ChatGPT? Stylistic Diffusion in Biomedical Abstracts in the LLM Era

Noor Haitham Saleem, Bernard J. Jansen
Machine Learning and Knowledge Extraction
Artificial Intelligence in Healthcare and Education
article

Are Scientists Starting to Sound like ChatGPT? Stylistic Diffusion in Biomedical Abstracts in the LLM Era

Noor Haitham Saleem, Bernard J. Jansen
article en

Abstract

Large language models are part of everyday scholarly writing, but their influence may not always appear as obvious AI authorship. More often, it may appear through gradual changes in academic style. This study examines whether biomedical abstracts show measurable signs of LLM-associated “same-ification” after the public release of ChatGPT. Instead of classifying individual abstracts as AI-written, our study analyses population-level changes across 28,415 PubMed abstracts published between 2018 and 2025. A strict dictionary of LLM-associated stylistic markers was used alongside a control vocabulary, lexical diversity, semantic similarity, interrupted time-series modelling, journal-level heterogeneity analysis, and a study-specific validation framework called SAME-ML. The findings show a clear post-2022 stylistic shift. Mean strict marker use increased from 4.995 to 11.658 markers per 1000 words, representing a 133.4% rise, while the control vocabulary slightly declined. This increase was strongest in 2024 and 2025 and was mainly driven by markers linked to value framing, contribution signalling, connective phrasing, improvement rhetoric, and problem-solution positioning. SAME-ML further showed that the post-ChatGPT period contained a learnable stylistic signal that remained visible after topic-masked and matched-sample validation. Overall, the findings show a corpus-level rhetorical shift and provide a reproducible approach for tracking stylistic change in scholarly communication over time.

Machine Learning and Knowledge ExtractionVol. 8(8)
Eastern Institute of Technology (NZ), Nankai University (CN)
Quality Education
Openalex Percentile: Top 12%
Artificial Intelligence in Healthcare and Education
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.