Resource-Aware Heterogeneous Cluster Scheduling System for Deep Neural Network Training Tasks

Deep Neural Networks (DNNs) have become integral to various modern artificial intelligence applications. As DNN models grow in complexity, diverse accelerators are increasingly utilized for training. Current heterogeneous cluster schedulers recognize performance disparities among models across different GPU types, managing costly GPU resources for users. However, these schedulers often neglect the significance of auxiliary resources in allocation decisions, relying on a GPU-based proportional distribution for CPU and memory resources. This study investigates the sensitivity of various DNN models to auxiliary resources through empirical experiments, revealing that training throughput for image and speech models is highly reliant on CPU allocation, while language models exhibit lower sensitivity. In response, we developed Nexus, a resource-aware scheduler designed to optimize resource utilization in heterogeneous GPU clusters. Experimental results demonstrate that Nexus can reduce average task completion time by up to 2.26x compared to traditional scheduling systems.