CPU Autoscaling With a Kernel of Truth

Cloud computing paradigms such as microservices and functions-as-a-service have made autoscaling an essential component of cloud application management. However, existing autoscalers struggle at capturing application dynamics, and have difficulties with precisely allocating quotas of shared system resources to the applications. We argue that one fundamental issue is the gap between native OS resource interfaces and surrogate user metrics that existing autoscalers use. In this paper, we take CPU autoscaling as an example: the cloud interface treats CPU resources as a percentage of the host CPU (e.g., millicore), while the OS kernel interprets CPU resources as time-shared quota slices allowed to run within a set period. We advocate for OS kernel support for CPU autoscaling to close the semantic gap, as it allows the autoscaler to perform precise, highly responsive resource allocation. We demonstrate the idea by developing Kscaler, a millisecond-scale CPU autoscaler for Linux. With kernel-level observability of fine-grained scheduler behavior, Kscaler outperforms state-of-the-art CPU autoscalers in responsiveness, precision, and efficiency while employing simple statistical methods.