Publication on the Pulsar Network, integrated in workflow management systems

Documentation of the design, deployment, and operationalization of the European Pulsar Network, developed within the EuroScienceGateway (ESG) project. This deliverable outlines how the Pulsar Network enables scalable, federated, and interoperable remote job execution across European Galaxy servers and compute infrastructures. This record showcases the technical architecture, automation strategies, and monitoring solutions behind the distributed execution framework, supporting reproducible workflows and efficient resource sharing. The network connects 13 Pulsar endpoints across 10 countries, integrated with six national Galaxy servers and the European Galaxy server. Deployments span public clouds, institutional HPCs, and EOSC resources, unified under a secure, open-source infrastructure stack using Terraform, Ansible, RabbitMQ, CVMFS, and SABER. The deliverable demonstrates how ESG addressed interoperability and scalability challenges through open infrastructure tooling, cross-institutional coordination, and continuous monitoring. It provides a replicable model for distributed compute resource integration and highlights Galaxy's extensibility in federated scientific computing. Project: EuroScienceGateway, funded by the European Union’s Horizon Europe programme (HORIZON-INFRA-2021-EOSC-01) under grant agreement number 101057388. Document: D3.2 Publication on the Pulsar Network, integrated in workflow management systems Work Package: Work Package 3. Pulsar Network: Distributed heterogeneuos compute Tasks: Task 3.1 Develop and maintain an Open Infrastructure-based deployment model for Pulsar endpoints Task 3.2 Add GA4GH Task Execution Service (TES) API to Pulsar Task 3.3 Build a European-wide network of Pulsar sites Task 3.4 Add TES support to WfExS (Workflow Execution Service) Task 3.5 Developing and maintaining national or domain-driven Galaxy servers Lead Beneficiary: CNRContributing Beneficiaries: CNR, INFN, ALU-FR, CNRS, CESNET, UiO, UB, EPFL, AGH/AGH-UST, BSC, VIB, IISAS, TUBITAK, UNIMAN

Institutions

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2025-08-07
DOI
https://doi.org/10.5281/zenodo.16744463
Primary Topic
Scientific Computing and Data Management
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Publication on the Pulsar Network, integrated in workflow management systems

Zenodo (CERN European Organization for Nuclear Research)
Scientific Computing and Data Management
article

Publication on the Pulsar Network, integrated in workflow management systems

article en

Abstract

Documentation of the design, deployment, and operationalization of the European Pulsar Network, developed within the EuroScienceGateway (ESG) project. This deliverable outlines how the Pulsar Network enables scalable, federated, and interoperable remote job execution across European Galaxy servers and compute infrastructures. This record showcases the technical architecture, automation strategies, and monitoring solutions behind the distributed execution framework, supporting reproducible workflows and efficient resource sharing. The network connects 13 Pulsar endpoints across 10 countries, integrated with six national Galaxy servers and the European Galaxy server. Deployments span public clouds, institutional HPCs, and EOSC resources, unified under a secure, open-source infrastructure stack using Terraform, Ansible, RabbitMQ, CVMFS, and SABER. The deliverable demonstrates how ESG addressed interoperability and scalability challenges through open infrastructure tooling, cross-institutional coordination, and continuous monitoring. It provides a replicable model for distributed compute resource integration and highlights Galaxy's extensibility in federated scientific computing. Project: EuroScienceGateway, funded by the European Union’s Horizon Europe programme (HORIZON-INFRA-2021-EOSC-01) under grant agreement number 101057388. Document: D3.2 Publication on the Pulsar Network, integrated in workflow management systems Work Package: Work Package 3. Pulsar Network: Distributed heterogeneuos compute Tasks: Task 3.1 Develop and maintain an Open Infrastructure-based deployment model for Pulsar endpoints Task 3.2 Add GA4GH Task Execution Service (TES) API to Pulsar Task 3.3 Build a European-wide network of Pulsar sites Task 3.4 Add TES support to WfExS (Workflow Execution Service) Task 3.5 Developing and maintaining national or domain-driven Galaxy servers Lead Beneficiary: CNRContributing Beneficiaries: CNR, INFN, ALU-FR, CNRS, CESNET, UiO, UB, EPFL, AGH/AGH-UST, BSC, VIB, IISAS, TUBITAK, UNIMAN

Zenodo (CERN European Organization for Nuclear Research)
University of Freiburg (DE), Institute of Biomembranes, Bioenergetics and Molecular Biotechnologies (IT)
Industry, innovation and infrastructure
Openalex Percentile: Top 55%
Scientific Computing and Data Management
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Publication on the Pulsar Network, integrated in workflow management systems · Zenodo (CERN European Organization for Nuclear Research) (2025) | TGRS Research Map | TGRS