Plan, Validate, Repeat: An Iterative Data Curation Framework for Ecologists

Presentation given at ESA 2025 in the FAIR and Open data track. For ecologists, data curation can be an especially difficult task because we study systems. This often involves collecting multiple types of data under difficult conditions, over long periods of time, or as part of a large team. All of these factors increase the need for robust data curation processes while also increasing the complexity of describing those processes and making them reproducible. My talk will be anchored by a narrative about a long term, multi-institution vector-borne disease research project that struggled to complete data curation tasks within the project timeline. I will argue that by employing four tactics – i) creating and maintaining a data management plan, ii) using data standards to guide the design of data storage and collection tools, iii) establishing programmatic data cleaning and validation processes early, and iv) having auditable and interpretable code and data stored in redundant locations – ecologists can greatly increase the accuracy, reproducibility, reliability, and quality of their data. These tactics focus on planning and prevention with the goal of identifying and fixing issues as early as possible, and being able to revert to past states if necessary. Using data standards front-loads much of the work around data documentation, while using scripted cleaning and validation workflows allows for precise communication of how data were curated. Through thoughtful planning, system design, and documentation, we can improve data quality and reliability in ecological science while also improving accuracy and reproducibility.