Exploring Multi-armed Bandits Learning Strategies in the Emergent Web Server

Self-adaptive systems are designed to modify their architecture orbehavior to uphold high-level objectives despite changes in theiroperating environments. A critical aspect of developing such systemsinvolves creating strategies to handle unexpected events inthe operating environments. While this remains an active area ofresearch within the autonomic computing and self-adaptive systemscommunity, one commonly adopted approach is leveragingmachine learning techniques, particularly reinforcement learning,to address unforeseen challenges. In this paper, we conduct experimentsusing the EmergentWeb Server exemplar, a publicly availableself-adaptive web server, to investigate various monitoring metricsand implement a multi-armed bandit reinforcement learningapproach. This approach enables the system to identify the optimalweb server configuration for maximizing performance undervarying workload patterns and operating conditions, enabling thesystem to react to unexpected events that rises from the operatingenvironment with minimum human interference.