The Observed Availability of Data and Code in Earth Science and Artificial Intelligence

Abstract As the use of artificial intelligence (AI) has grown exponentially across a wide variety of science applications, it has become clear that it is critical to share data and code to facilitate reproducibility and innovation. AMS recently adopted the requirement that all papers include an availability statement. However, there is no requirement to ensure that the data and code are actually freely accessible during and after publication. Studies show that without this requirement, data are openly available in about a third to a half of journal articles. In this work, we surveyed two American Meteorological Society (AMS) journals, Artificial Intelligence for the Earth Systems (AIES) and Monthly Weather Review (MWR), and two non-AMS journals. These journals varied in primary topic foci, publisher, and requirement of an availability statement. We examined the extent to which data and code are stated to be available in all four journals and if readers could easily access the data and code. Our analysis found that roughly 75% of all articles that produced data and had an availability statement made at least some of their data openly available. Code was made openly available less frequently in three out of the four journals examined. Access was inhibited to data or code in approximately 15% of availability statement that contained at least one link. Common justifications for not making data or code openly available referenced dataset size and restrictions of availability from non-coauthor entities. Finally, several recommendations are made for improving data and code availability. Significance Statement Making code and data available to future researchers is critical for research reproducibility and to further innovation. Despite this, if it is not required, authors share their code and data only about one-third to one-half of the time. We show that even with the new AMS journal requirement to include an availability statement, the actual availability is limited. This issue is important to address for future research, especially with the growing amount of research in AI. If data and models are made easily available, people can innovate on these models in a more equitable manner.