Dissecting Corpora Construction: The Case of Environmental Social Science

The use of computational techniques to understand large text datasets has become commonplace. At the same time, the idea that the output generated from computational analyses is only as good as the data that was input into them has become ubiquitous. Yet the consequences of decision-making during the corpora construction process have not been adequately examined. In this paper, we dissect corpora construction using the case of environmental social science literature. Specifically, we ask about the consequences of including (or excluding) a massive publisher of megajournals on the overall structure of a carefully identified environmental social science database from 2008-2022 (n=166,804 articles). Results of a multi-pronged bibliometric analysis, including both topic models and structural topic models, indicate that the articles published by massive publishers exhibit some variation in subject matter as well as minor differences in content and style, including citation practices. Conclusions draw attention to the importance of dissecting corpora construction in an age of proliferating science.