Dynamic Resource Allocation of Virtualized GPUs and CPUs for Scalable AI Workloads in Containerized Environments

Resource allocation for virtualized GPUs and CPUs in containerized environments supporting dynamic AI workloads is of prime importance. Traditional dynamic approaches miserably fail in case of sudden spikes in workload, while static allocation is inefficient overall. The paper proposes a unique approach by using a hybrid predictive scaling technique combining time-series analysis with reinforcement learning. The framework optimizes GPU and CPU provisioning in a Kubernetes-based architecture with a focus on workload analysis and real-time contention monitoring. Thorough testing on a variety of AI workloads compared to current methodologies shows substantial improvements in latency, energy efficiency, resource usage, and throughput. This framework effectively handles scaling issues in cloud and edge contexts and will enable future AI systems to intelligently and scalable manage their resources.