Energy-Aware DNN Task Scheduling with Dynamic Batching and Frequency Adjustment

This work presents an energy-aware scheduling algorithm for deep neural network (DNN) inference tasks under energy, memory, and deadline constraints. By leveraging a precomputed lookup table (LUT) of performance metrics, the scheduler dynamically optimizes batch sizes, selectively applies concurrency, and adjusts GPU frequency as a last resort. The proposed approach maximizes batch sizes up to an empirically determined threshold and executes tasks concurrently when task deadlines permit and memory constraints allow. GPU frequency scaling is applied only when neither batch size optimization nor concurrency can ensure deadline compliance. Experiments on CIFAR-10 and CIFAR-100 demonstrate the scheduler's ability to meet deadlines while minimizing energy consumption.