Generative AI - Signal to Noise

Abstract The sudden deployment of large language models (LLMs) has been a seismic event for science, with professional scientists, including biologists, struggling to work out how to fit this new technology into their working lives. The benefits of LLMs are manifold but here we flag a neglected and very serious negative aspect of their use in the area of culturomics. This field depends on analysing word frequencies to pick out the prevailing zeitgeist in corpora of text that are readily available online through social media and analysable through modern software. This provides insights into human culture on a scale that was impossible 20 years ago. Culturomics has influenced many topics where understanding the human perspective is key. However, LLMs are ‘polluting the waters’ by producing AI generated text that is, by definition, not what people are talking about. We believe there’s a strong case to be made for highlighting the nature of LLM pollution and give our view for how to clean the waters.