Reinforcement Learning for Efficient Multi-phase Resource Allocation

Efficient resource allocation is pivotal for achieving high performance in emerging computer systems, where multiple users and tasks compete for shared resources. This challenge spans various domains, including data centers, multicore processors, cloud computing and edge computing, each requiring nuanced allocation strategies to balance competing demands. Traditional approaches often assume concave utility (performance) functions for users, simplifying optimization but failing to capture the complexities of real-world scenarios where non-concave utility functions prevail. Numerous works in the literature apply the greedy algorithm to nonconcave utility functions, resulting in suboptimal solution due to the short-sighted behaviors. To improve this gap, we propose a novel multiphase resource allocation framework that accurately reflects the non-linear dynamics of these systems. To tackle the NP-complete nature of this problem, we formulate a customized resource allocation Markov Decision Process (MDP) that integrates the characteristics of multi-phase utility functions into a nuanced design of the key MDP components, such as state representations, reward signals, and actions. We explore two reinforcement learning (RL)-based methods, specifically Dueling Deep Q-Network (Dueling DQN) and Proximal Policy Optimization (PPO), to optimize resource allocation over time. Our RL-based strategies outperform the conventional greedy algorithm by approximately 37% in standard environments and up to 73% in specialized environments, highlighting their effectiveness in handling the resource allocation problem with non-concave utility functions and achieving scalable, real-time solutions.