Hierarchical Reinforcement Learning for Energy-Efficient API Traffic Optimization in Large-Scale Advertising Systems

Digital advertising infrastructure represents a substantial component of global computing resources, with significant environmental impact due to its massive energy consumption and carbon footprint. This work addresses the sustainability challenges posed by inefficient API traffic management in large-scale advertising systems, where conventional static approaches lead to resource overprovisioning and energy waste. We propose AdaptiveGate, a sustainability-oriented hierarchical reinforcement learning framework that dynamically optimizes API traffic flows to enhance resource efficiency and reduce environmental impact. The proposed methodology employs a constrained Markov Decision Process formulation with a multi-objective reward function explicitly designed to balance system performance with resource efficiency. Our framework implements a two-tier architecture of twin delayed Deep Deterministic Policy Gradient agents: global agents minimize cross-datacenter energy expenditure through intelligent traffic routing, while local agents maximize resource utilization through service-specific load balancing. Empirical evaluation on production advertising systems processing over 2.5 million requests per second reveals significant sustainability improvements: 42.3% reduction in tail latency, 35.7% increase in throughput, and 18% decrease in overall energy consumption compared to state-of-the-art methods. The system demonstrates exceptional adaptability across diverse traffic conditions and operational scales, providing compelling evidence that AI-driven methods can substantially improve digital infrastructure sustainability. This work contributes to sustainable computing by establishing a framework that optimizes computational resource allocation, minimizes energy waste, and advances environmentally responsible high-performance computing systems, aligning with multiple Sustainable Development Goals including responsible consumption and affordable clean energy.