Does corpus size influence normalised frequencies?
Abstract Several frequency-based measures are influenced by corpus size (e.g. lexical diversity or text similarity measures). It is largely unquestioned, however, that normalised frequencies correct for the influence of corpus size – but it has not yet been systematically tested whether and how they might be influenced by corpus size themselves. The central question is whether the normalised frequency of an element in a smaller corpus can be meaningfully compared to the normalised frequency of the same element in a larger corpus. We are testing the association between lists of normalised frequencies derived from corpus samples of different sizes from six languages. Our results suggest that the size of the underlying corpora does not negatively influence comparisons of normalised frequency lists, i.e. different corpus sizes do not lead to normalised frequencies no longer being comparable. For lower-frequency types, these associations decrease rather quickly. These empirical findings converge with predictions from statistical theory.
Authors
- Sascha Wolfer (ORCID: https://orcid.org/0000-0002-8893-8153)
- Alexander Koplenig (ORCID: https://orcid.org/0000-0002-9630-9680)
Institutions
- Leibniz Institute for the German Language (DE)
Publication Details
- Journal
- Corpus Linguistics and Linguistic Theory
- Published
- 2025-06-24
- DOI
- https://doi.org/10.1515/cllt-2024-0040
- Primary Topic
- Language and cultural evolution
- Type
- article
- Field-Weighted Citation Impact
- 0.00