Breaking the Complexity Barrier: Enhancing Quality of Service in Simultaneous Multithreading Processors

Within the wide variety of applications running in the cloud, demands for service quality and performance guarantees need special attention. Since cloud nodes typically utilize Simultaneous Multithreading (SMT) cores, processor resources may need to be shared and managed among multiple applications. A customer application that demands high levels of Quality of Service (QoS) may require more resources than its best-effort allocated share. QoSMT is an earlier study that focuses on 2-thread SMT cores to address this issue. While a simple relationship between two threads may be relatively straightforward to manage, the interactions among multiple threads can quickly become highly complex and difficult to design and maintain. In this paper, we extend the original QoSMT study to 4-thread SMT cores by proposing three alternative scheduling algorithms that regulate resource usage between a high-priority thread and three ordinary batch-type threads. On average, we achieve 7.2% better overall throughput and up to 218% better throughput for the latency-sensitive, high-priority thread. More than 92% of 600 workloads display higher performance than the baseline configuration, and 8% of workloads perform ten times better compared to a traditional 4-thread baseline SMT with unmanaged resources. • The original 2-thread QoSMT study is extended for a 4-thread QoSMT processor, where the complexity of relations among threads increases significantly. • To address this complexity of resource sharing among threads, we implement three resource distribution algorithms (RR, UB1, and UB2) with sophisticated optimizations uniquely tailored for the 4-thread QoSMT processor context. • We focus on three major metrics (the throughput, the fairness, and the QoS of the latency-sensitive thread) to evaluate our proposed algorithms compared to a baseline SMT processor. • We conduct a deviation analysis to verify the prediction accuracy of our proposed heuristic algorithms. • Finally, we also elaborate on the hardware complexity and power overhead of the proposed algorithms.