Hybrid HPC: can we finally manage our data easily and consistently between on-prem and out in the cloud?

Public cloud use in a hybrid HPC environment remains a challenge: compute is easy, but data access comes with complications. Storage vendors have implemented various approaches to the problem of bursting data to the cloud. Some use global namespaces to stretch the filesystem others use snapshots and others shared repositories. In this talk we explore these different approaches: their strengths, weaknesses and scenarios where they best apply.

Institutions

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2024-10-30
DOI
https://doi.org/10.5281/zenodo.15293601
Primary Topic
semigroups and automata theory
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Hybrid HPC: can we finally manage our data easily and consistently between on-prem and out in the cloud?

Zenodo (CERN European Organization for Nuclear Research)
semigroups and automata theory
article

Hybrid HPC: can we finally manage our data easily and consistently between on-prem and out in the cloud?

article en

Abstract

Public cloud use in a hybrid HPC environment remains a challenge: compute is easy, but data access comes with complications. Storage vendors have implemented various approaches to the problem of bursting data to the cloud. Some use global namespaces to stretch the filesystem others use snapshots and others shared repositories. In this talk we explore these different approaches: their strengths, weaknesses and scenarios where they best apply.

Zenodo (CERN European Organization for Nuclear Research)
Xenon Pharmaceuticals (Canada) (CA)
Openalex Percentile: Top 56%
semigroups and automata theory
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Hybrid HPC: can we finally manage our data easily and consistently between on-prem and out in the cloud? · Zenodo (CERN European Organization for Nuclear Research) (2024) | TGRS Research Map | TGRS