Supporting FAIR Scientific Workflows with the JGI Analysis Workflow Service (JAWS)

The JGI Analysis Workflow Service (JAWS) is a scalable and centralized workflow execution framework developed by the U.S. Department of Energy Joint Genome Institute (JGI, jgi.doe.gov). It enables researchers to efficiently run complex computational workflows across multiple high-performance computing (HPC) resources. JAWS automates resource configuration, data transfer, and job scheduling, allowing users to focus on their scientific tasks while ensuring workflow reproducibility and scalability.JAWS takes advantage of robust software packages, Docker containers, and the Workflow Description Language (WDL) as building blocks for standardized, Findable, Accessible, Interoperable, Reusable (FAIR) workflows. Over the past five years, we have collaborated with users at the JGI and the National Microbiome Data Collaborative (NMDC) to execute their pipelines on resources at the National Energy Research Scientific Computing Facility (NERSC), Pacific Northwest National Laboratory (PNNL), and on JGI-owned resources at Lawrence Berkeley National Laboratory (LBNL). The model of having a workflow protocol (WDL), software environment (Docker container), and input data has worked incredibly well for users. JAWS has been able to scale to support thousands of jobs, and the effort to standardize the workflows has yielded significant performance improvements. The software is open-source and available under the BSSD license.