A Modest Rise That an Exploratory Year-Level Test Does Not Separate From Noise: LLM-Associated Vocabulary in the US Federal Register, 2015-2026

Excess-vocabulary studies found sharp post-2022 rises in words favoured by large language models (LLMs) in scientific abstracts and other genres; for the US Federal Register, a pilot study found AI-text classifier scores near zero on 386 AI-related documents. This paper applies that design, specified in an unpublished pre-registration whose timing cannot be verified, to the preambles (agencies' explanatory statements) of 326,374 rules, proposed rules and notices from January 2015 to September 2026 (600 million tokens). The registered primary quantity is the 2024 ratio of a 13-family marker set (such as ``delve'', ``notably'') to its 2015-2022 log-linear trend, divided by the same ratio for 65 frequency-matched control words chosen from 2015-2019 data. It is 1.52, the largest of twelve years. Its registered 95% day-bootstrap interval (1.25 to 1.88) lies above 1, but the registered placebo in time shows this interval is anti-conservative: fitting the trend on 2015-2018 yields equally large ``effects'' for 2020 (1.52, 1.11 to 2.15) and 2021 (1.67, 1.06 to 2.66), before ChatGPT's release. The registered test is therefore not taken as support. An exploratory year-level prediction interval, which does not reject in either placebo year but whose calibration is not established, does not separate the 2024 excess from baseline variation (two-sided p = 0.09); without control words mechanically flagged as likely names or non-lexicon words, D_2024 = 1.48 (p = 0.08). A registered meeting-notice placebo also failed, on too few tokens to inform. Marker rates are low (0.37 per 10,000 tokens in 2024). 2024 point estimates exceeded 2 in five of the twelve largest agencies; agency differences were not tested and agency intervals share the day bootstrap's miscalibration. The evidence is consistent with a limited LLM-associated style shift in federal regulatory prose but does not establish it.

Authors

Institutions

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-10-27
DOI
https://doi.org/10.5281/zenodo.23699304
Primary Topic
Computational and Text Analysis Methods
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
preprint

A Modest Rise That an Exploratory Year-Level Test Does Not Separate From Noise: LLM-Associated Vocabulary in the US Federal Register, 2015-2026

Keisuke Watanabe
1 links
Zenodo (CERN European Organization for Nuclear Research)
Computational and Text Analysis Methods
preprint

A Modest Rise That an Exploratory Year-Level Test Does Not Separate From Noise: LLM-Associated Vocabulary in the US Federal Register, 2015-2026

Keisuke Watanabe
preprint en

Abstract

Excess-vocabulary studies found sharp post-2022 rises in words favoured by large language models (LLMs) in scientific abstracts and other genres; for the US Federal Register, a pilot study found AI-text classifier scores near zero on 386 AI-related documents. This paper applies that design, specified in an unpublished pre-registration whose timing cannot be verified, to the preambles (agencies' explanatory statements) of 326,374 rules, proposed rules and notices from January 2015 to September 2026 (600 million tokens). The registered primary quantity is the 2024 ratio of a 13-family marker set (such as ``delve'', ``notably'') to its 2015-2022 log-linear trend, divided by the same ratio for 65 frequency-matched control words chosen from 2015-2019 data. It is 1.52, the largest of twelve years. Its registered 95% day-bootstrap interval (1.25 to 1.88) lies above 1, but the registered placebo in time shows this interval is anti-conservative: fitting the trend on 2015-2018 yields equally large ``effects'' for 2020 (1.52, 1.11 to 2.15) and 2021 (1.67, 1.06 to 2.66), before ChatGPT's release. The registered test is therefore not taken as support. An exploratory year-level prediction interval, which does not reject in either placebo year but whose calibration is not established, does not separate the 2024 excess from baseline variation (two-sided p = 0.09); without control words mechanically flagged as likely names or non-lexicon words, D_2024 = 1.48 (p = 0.08). A registered meeting-notice placebo also failed, on too few tokens to inform. Marker rates are low (0.37 per 10,000 tokens in 2024). 2024 point estimates exceeded 2 in five of the twelve largest agencies; agency differences were not tested and agency intervals share the day bootstrap's miscalibration. The evidence is consistent with a limited LLM-associated style shift in federal regulatory prose but does not establish it.

Zenodo (CERN European Organization for Nuclear Research)
PPG Industries (United States) (US), Institute of Glass (RU), Central Glass (Japan) (JP)
Computational and Text Analysis Methods
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon