Comparative Study on DQN and PPO for Cloud Resource Optimization

Cloud computing has become the backbone of mod- ern digital infrastructure, supporting millions of applications that demand high performance, scalability, and cost efficiency. However, dynamic workloads and heterogeneous resources continue to challenge the design of adaptive resource management systems. Deep Reinforcement Learning (DRL) offers a promising paradigm by enabling autonomous, data-driven decision- making based on continuous environmental feedback. This review paper systematically examines the application of DRL algorithms particularly Deep Q-Network (DQN) and Proximal Policy Optimization (PPO)—in optimizing cloud resource allocation, load balancing, and energy efficiency. The survey categorizes research advancements into value-based, policy-gradient, and hybrid learning architectures and analyzes their comparative strengths across diverse scenarios such as auto scaling, container placement, and dynamic job scheduling. It further explores recent strategies like multi-agent systems, federated DRL, and energy- aware reinforcement frameworks aimed at achieving sustainable cloud operations. Concluding insights identify current challenges, including convergence stability, reward modeling, and cross- environment generalization, while outlining promising directions for integrating DRL with edge computing, green AI, and real- time orchestration technologies.