DISTRI: Development and Integration of Simulation Tools for Resilient Infrastructure
In contemporary scientific research, data acquisition and analysis platforms have grown increasingly complex, often spanning multiple facilities with diverse internal structures. Efficiently managing the interactions between job scheduling, resource allocation, and networking across these distributed systems requires a robust simulation framework. However, existing simulators fall short in capturing the detailed interactions necessary for comprehensive analysis of large-scale distributed environments. To address this gap, we introduce DISTRI, a versatile framework specifically designed for the development and testing of distributed multi-facility workflows. DISTRI allows for customizable facility configurations and includes built-in support for distributed, resilient scheduling and resource management, alongside detailed network simulation for data communication. Key features of DISTRI encompass inter- and intra-facility resource management, agent-based distributed scheduling, and extensive performance metrics logging for both resource and network management. By providing these essential tools, DISTRI enables thorough analysis and optimization, thereby advancing research in the resilience and efficiency of multi-facility systems.
Authors
- Komal Thareja (ORCID: https://orcid.org/0000-0002-8113-6967)
- Ewa Deelman (ORCID: https://orcid.org/0000-0001-5106-503X)
- Imtiaz Mahmud (ORCID: https://orcid.org/0000-0002-3390-2698)
- Kesheng Wu (ORCID: https://orcid.org/0000-0002-6907-3393)
- Anirban Mandal (ORCID: https://orcid.org/0000-0001-5145-8618)
- Cong Wang (ORCID: https://orcid.org/0000-0002-2603-4626)
- Mariam Kiran (ORCID: https://orcid.org/0000-0001-5775-0855)
- Krishnan Raghavan (ORCID: https://orcid.org/0000-0001-9409-2011)
- Paweł Żuk (ORCID: https://orcid.org/0000-0001-5293-8517)
Institutions
- University of Southern California (US)
- Argonne National Laboratory (US)
- Oak Ridge National Laboratory (US)
- Lawrence Berkeley National Laboratory (US)
- Renaissance Computing Institute (US)
Publication Details
- Published
- 2024-12-15
- DOI
- https://doi.org/10.1109/bigdata62323.2024.10825783
- Citations
- 2
- Primary Topic
- Simulation Techniques and Applications
- Type
- article
- Field-Weighted Citation Impact
- 0.98
Funders
- U.S. Department of Energy