Efficient Resource Management for Real-Time AI Systems in the Cloud using Reinforcement Learning

The advent of artificial intelligence (AI) has driven an emergence in applications demanding online responses to immense amounts of data. Typically, the cloud-based deployment of these applications requires resource management to be optimized for high performance while meeting cost efficiency. However, conventional static resource allocation methods may only partially apply to the context of real-time AI applications as they are dynamic, unpredictable, and require more efficient & adaptive ways of allocating resources. To address this challenge, and as a solution in this research, we introduce an RL-based method for online resource management of cloud-based AI systems. Reinforcement learning (RL) is a machine-learning algorithm that allows systems to learn how best to perform tasks in changing environments based on feedback from those interactions with the environment. The research utilizes reinforcement learning to enable dynamic resource allocation, ensuring that resources are provisioned on demand rather than relying on static allocation methods defined at design time.