Advancing open and reproducible water data science by integrating data analytics with an online data repository

Scientific and management challenges in the water domain require synthesis of diverse data. Many analysis tasks are difficult because datasets are large and complex, standard formats are not always agreed upon or mapped to efficient data structures, scientists may lack training for tackling large and complex datasets, and it can be difficult to share and reproduce data science workflows. Overcoming barriers to accessing, organizing, and preparing datasets for analyses can transform how water scientists work. Building on the HydroShare repository’s cyberinfrastructure, we created a Python package that automates data retrieval, organization, and curation for analysis, reducing time spent in choosing appropriate data structures and writing data ingestion code. It manages metadata and automates data loading into performant structures consistent with Python’s visualization, analysis, and data science capabilities and can be used to build and share more reproducible scientific workflows in HydroShare following FAIR (Findable, Accessible, Interoperable, and Reusable) principles. • The hsclient python client library was created for the HydroShare repository. • hsclient automates data retrieval, organization, and curation for analysis. • Integrating hsclient with performant data structures speeds time to analysis. • hsclient enables building and sharing more reproducible scientific workflows. • hsclient can be used in any Python environment, including online JupyterHubs.