Failure-Aware Resource Allocation for Reliable Distributed Machine Learning Training

Long-running distributed training is exposed to worker failures and checkpoint overhead, yet lightweight GPU allocation rules often optimize failure-free throughput. This paper presents FARAS, a failure-aware resource allocation strategy that jointly selects checkpoint periods and allocates GPUs using estimated recoverable training progress. FARAS combines a per-worker failure hazard, nonlinear parallel speedup, checkpoint duration, and recovery delay in a marginal-goodput score. A minute-resolution Monte Carlo simulator evaluates five schedulers on synthetic batches of 14 training jobs sharing 24 GPUs, with 12 independently generated workload instances per condition. In the baseline failure setting, FARAS achieves a mean job completion time of 1,947.9 simulated minutes, compared with 2,037.8 for equal allocation with fixed checkpoints and 1,976.1 for size-aware allocation with adaptive checkpoints. These correspond to reductions of 4.41% and 1.42%, respectively. Across the tested failure intensities, FARAS consistently gives the lowest mean completion time among the examined policies. An additional paired experiment evaluates the impact of twofold underestimation and overestimation of worker failure hazards. The reported results describe synthetic system behavior rather than hardware measurements.

Authors

Institutions

Publication Details

Journal
Academic Journal of Applied Sciences
Published
2026-11-16
DOI
https://doi.org/10.54097/xqj1gs27
Primary Topic
Cloud Computing and Resource Management
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Failure-Aware Resource Allocation for Reliable Distributed Machine Learning Training

Eva van Dijk, Matthijs de Vries
Academic Journal of Applied Sciences
Cloud Computing and Resource Management
article

Failure-Aware Resource Allocation for Reliable Distributed Machine Learning Training

Eva van Dijk, Matthijs de Vries
article en

Abstract

Long-running distributed training is exposed to worker failures and checkpoint overhead, yet lightweight GPU allocation rules often optimize failure-free throughput. This paper presents FARAS, a failure-aware resource allocation strategy that jointly selects checkpoint periods and allocates GPUs using estimated recoverable training progress. FARAS combines a per-worker failure hazard, nonlinear parallel speedup, checkpoint duration, and recovery delay in a marginal-goodput score. A minute-resolution Monte Carlo simulator evaluates five schedulers on synthetic batches of 14 training jobs sharing 24 GPUs, with 12 independently generated workload instances per condition. In the baseline failure setting, FARAS achieves a mean job completion time of 1,947.9 simulated minutes, compared with 2,037.8 for equal allocation with fixed checkpoints and 1,976.1 for size-aware allocation with adaptive checkpoints. These correspond to reductions of 4.41% and 1.42%, respectively. Across the tested failure intensities, FARAS consistently gives the lowest mean completion time among the examined policies. An additional paired experiment evaluates the impact of twofold underestimation and overestimation of worker failure hazards. The reported results describe synthetic system behavior rather than hardware measurements.

Academic Journal of Applied SciencesVol. 3(3)
University of Amsterdam (NL)
Cloud Computing and Resource Management
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon