Trade-Off between Performance and Power Consumption in Deep Learning Training

The rapid advancement of deep learning technologies has transformed various sectors, enabling sophisticated applications across industries. However, training deep learning models is an energy-intensive process, leading to significant power consumption and associated costs. This paper investigates the inherent trade-off between power consumption and average waiting time in systems managing multiple job classes during model training. Recognizing that not all training jobs are created equal, we propose a systematic approach to optimize power usage by adjusting GPU power limits based on job urgency. Leveraging matrix geometric approximation, we develop an algorithm to compute mean sojourn times and average power consumption, demonstrating its effectiveness through experimental validation and simulations. Our findings reveal the existence of a power-performance trade-off, offering valuable insights for practitioners seeking to balance computational efficiency with energy sustainability in deep learning workflows.