Using Globus as the high-performance data transport mechanism at Monash University
Current in-house solutions have been pivotal in Monash University's data storage and transfer infrastructure; however, the data management requirements have outgrown currently used systems, particularly in their ability to provide data reliable ingest from the instruments generating large datasets and large volumes of data, ability to easily move the data between the storage and computing facilities, to automate the data processing pipelines, and easily share and collaborate on data. We ve been investigating various data orchestration solutions and deployed Globus as a pilot solution of choice. We report on our journey in trialling Globus. We will share our experience, challenges, and will discuss the deployed solutions across the Monash network and Monash eResearch infrastructure.
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2024-10-30
- DOI
- https://doi.org/10.5281/zenodo.15293628
- Primary Topic
- Distributed and Parallel Computing Systems
- Type
- article
- Field-Weighted Citation Impact
- 0.00