Efficient Resource Management for Real-time AI Systems in the Cloud using Reinforcement Learning

The advent of artificial intelligence (AI) has driven an emergence in applications demanding online responses to immense amounts of data. Typically, the cloud-based deployment of these applications requires resource management to be optimized for high performance while meeting cost efficiency. However, conventional static resource allocation methods may only partially apply to the context of real-time AI applications as they are dynamic, unpredictable, and require more efficient & adaptive ways of allocating resources. To address this challenge, and as a solution in this research, we introduce an RL-based method for online resource management of cloud-based AI systems. Reinforcement learning ($\mathbf{R L}$) is a machine-learning algorithm that allows systems to learn how best to perform tasks in changing environments based on feedback from those interactions with the environment. The third way our proposed approach utilizes the weapon of RL entails allowing for dynamic resource allocation so that resources are allocated when needed rather than from a static method as set at design time.