Flux Emulator: First Insights into Optimizing Scheduling for Exascale HPC
El Capitan, currently the world's largest supercomputer at 1.742 Ex-aflop/s, introduces challenges in scheduling due to its scale and innovative rabbit nodes, which traditional schedulers cannot efficiently handle. Flux, a resource and job management system, handles dynamic resource allocation tailored for exascale systems through its graph-based scheduler, Fluxion. This work introduces the Flux Emulator, a tool designed to test scheduling policies in Fluxion without impacting production systems. The emulator plugs into the real components of Flux and Fluxion to mimic job execution, emulate resource usage, and collect information on how the job behaves. Preliminary tests show negligible overhead introduced by the emulator and demonstrate its effectiveness in evaluating scheduli ng policies, like conservative backfilling, in a fraction of the time required with a real system.
Authors
- Dewi Yokelson (ORCID: https://orcid.org/0000-0003-1453-5906)
- Stephanie Brink (ORCID: https://orcid.org/0000-0002-1458-8453)
- Olga Pearce (ORCID: https://orcid.org/0000-0002-1904-9627)
- Daniel Milroy (ORCID: https://orcid.org/0000-0001-6500-3227)
- Michela Taufer (ORCID: https://orcid.org/0000-0002-0031-6377)
- Tapasya Patki (ORCID: https://orcid.org/0000-0003-2543-9688)
- Tom Scogland (ORCID: https://orcid.org/0000-0001-7234-5743)
- Ian Lumsden (ORCID: https://orcid.org/0000-0003-0009-5487)
- W. Jay Ashworth (ORCID: https://orcid.org/0009-0006-8395-3529)
Institutions
- Lawrence Livermore National Laboratory (US)
- Knoxville College (US)
- University of Tennessee at Knoxville (US)
Publication Details
- Published
- 2025-07-20
- DOI
- https://doi.org/10.1145/3731545.3735121
- Primary Topic
- Distributed and Parallel Computing Systems
- Type
- article
- Field-Weighted Citation Impact
- 0.00