Testing English News Articles for Lexical Homogenization Due to Widespread Use of Large Language Models

It is widely assumed that Large Language Models (LLMs) are shaping language, with multiple studies noting the growing presence of LLM-generated content and suggesting homogenizing effects.However, it remains unclear if these effects are already evident in recent writing.This study addresses that gap by comparing two datasets of English online news articles -one from 2018, prior to LLM popularization, and one from 2024, after widespread LLM adoption.We define lexical homogenization as a decrease in lexical diversity, measured by the MATTR, Maas, and MTLD metrics, and introduce the LLM-Style-Word Ratio (SWR) to measure LLM influence.We found higher MTLD and SWR scores, yet negligible changes in Maas and MATTR scores in 2024 corpus.We conclude that while there is an apparent influence of LLMs on written online English, homogenization effects do not show in the measurements.We therefore propose to apply different metrics to measure lexical homogenization in future studies on the influence of LLM usage on language change.

Authors

Institutions

Publication Details

Published
2025-01-01
DOI
https://doi.org/10.18653/v1/2025.acl-srw.95
Primary Topic
Natural Language Processing Techniques
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Testing English News Articles for Lexical Homogenization Due to Widespread Use of Large Language Models

Sarah Fitterer, Dominik Gangl, Jannes Ulbrich
Natural Language Processing Techniques
article

Testing English News Articles for Lexical Homogenization Due to Widespread Use of Large Language Models

Sarah Fitterer, Dominik Gangl, Jannes Ulbrich
article en

Abstract

It is widely assumed that Large Language Models (LLMs) are shaping language, with multiple studies noting the growing presence of LLM-generated content and suggesting homogenizing effects.However, it remains unclear if these effects are already evident in recent writing.This study addresses that gap by comparing two datasets of English online news articles -one from 2018, prior to LLM popularization, and one from 2024, after widespread LLM adoption.We define lexical homogenization as a decrease in lexical diversity, measured by the MATTR, Maas, and MTLD metrics, and introduce the LLM-Style-Word Ratio (SWR) to measure LLM influence.We found higher MTLD and SWR scores, yet negligible changes in Maas and MATTR scores in 2024 corpus.We conclude that while there is an apparent influence of LLMs on written online English, homogenization effects do not show in the measurements.We therefore propose to apply different metrics to measure lexical homogenization in future studies on the influence of LLM usage on language change.

Technische Universität Berlin (DE)
Quality Education
Openalex Percentile: Top 90%
Natural Language Processing Techniques
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Testing English News Articles for Lexical Homogenization Due to Widespread Use of Large Language Models — Sarah Fitterer, Dominik Gangl, et al. · (2025) | TGRS Research Map | TGRS