How Climate Model Developers Deal With Bugs

Abstract General circulation models (GCMs) are not only powerful tools to understand Earth's climate system and to forecast the weather. They are also large software programs written by humans. As such, they contain coding mistakes, so‐called bugs. Researchers communicate results generated with GCMs and document new model versions, but seldom explicitly communicate the bugs they find in their models, let alone the practices surrounding them. This study portrays practices around bugs that were found during recent ICON development, and the workflow from getting a suspicion to fixing and communicating the bug. Eleven qualitative in‐depth interviews were conducted with domain scientists and scientific programmers involved in ICON development. The interviews detail the workflow for dealing with bugs, highlighting that it is only partly standardized. For example, scientific testing is complicated by the fact that there is no absolute truth in terms of results that the model could be tested against. Thus testing resists standardization, so that dealing with bugs remains a laborious process. Being confronted and dealing with bugs, modelers aim for a model that is “good enough” rather than perfect. This stance is pragmatic and relaxes exuberant expectations for GCMs, especially considering their bugs. However, the goal of “good enough” is troubling with regard to GCMs' use as universal tools, with high societal stakes. Who decides that the model is “good enough,” and what for?