Reinforcement Learning Algorithms for Adaptive Load Balancing in Publish/Subscribe Systems: PPO, UCB, and Epsilon-Greedy Approaches

This research addresses load balancing challenges in publish/subscribe (Pub/Sub) systems through a comprehensive exploration of reinforcement learning (RL) techniques. Algorithms (epsilon-greedy, Upper Confidence Bound, round-robin, and least connections) were evaluated to establish baseline performance metrics. Building on this foundation, we develop enhanced versions of epsilon-greedy and UCB algorithms tailored to the Pub/Sub context. Additionally, we introduce a custom approach utilizing Proximal Policy Optimization (PPO) to learn adaptive load-balancing policies. Our work provides a thorough comparative analysis of diverse RL methods, offering insights into their strengths and weaknesses in optimizing Pub/Sub system performance. Experimental results demonstrate the potential of RL, particularly our developed algorithms, to significantly improve latency, throughput, and overall system efficiency compared to traditional load-balancing strategies. Notably, the PPO-based approach exhibits superior performance during burst traffic and failure scenarios, highlighting its resilience and adaptability in dynamic environments.