Reinforcement Learning-based Cloud Autoscaler Initialization via Evolutionary Algorithms

Scientific workflows are a popular tool for running experiments and simulations. Moreover, Cloud infrastructures are widely used to execute workflows since they can handle elastic demand of multiple resources. However, this requires an effective autoscaling strategy to guarantee good performance and low cost. Reinforcement Learning (RL) has been used in the past to obtain efficient autoscalers. Nevertheless, RL approaches often exhibit poor initial performance, a problem than can hurt overall performance. We present a novel two-phase autoscaling strategy that combines Evolutionary Computation (EC) and RL. The proposal ensembles and provides a pre-optimized initial policy, obtained offline via EC, to an existing RL-based online autoscaler. The strategy aims to improve the overall performance during the learning process reducing makespan and execution cost.