The Cloud-based Workflow Manager (CloWM) - An integrated platform for highly scalable workflow execution

Data analysis in life sciences often suffers from reproducibility, standardization, and accessibility issues due to the reliance on custom CLI scripts and the lack of easily accessible compute resources. The Cloud-based Workflow Manager (CloWM) addresses these challenges by providing a user-friendly platform for executing complex data analysis workflows in a standardized and reproducible manner. The system leverages cloud-based infrastructure to offer highly-scalable compute resources and personalized data storage through sharable S3 buckets. By encapsulating versioned Nextflow workflows as web services, developers can easily share and execute their analysis pipelines. Moreover, to facilitate efficient data management, workflow developers can upload and manage persistent and versioned workflow resources - like large, static databases - in CloWM. These are then accessible from within workflow executions, thus eliminating redundant data transfers and reducing workflow execution time. A robust authentication and authorization system, integrated with the LifeScience AAI SSO solution, ensures secure personalized access and data protection. An intuitive web interface which supports the automatic generation of workflow parameter interfaces, streamlines workflow management and execution. In addition, a REST API secured by API tokens allows programmatic interaction with the system, enhancing its flexibility and integration capabilities. CloWM empowers researchers to focus on scientific insights by handling and hiding the complexities of cloud-based infrastructure provisioning and distributed data management while workflow developers benefit from a broader target audience that can easily execute their workflows in a highly-scalable fashion. The CloWM system is available under https://clowm.bi.denbi.de and the code can be found at https://gitlab.ub.uni-bielefeld.de/cmg/clowm.