dStream: An Online-Based Dynamic Operator-Level Query Mapping Scheme on Discrete CPU-GPU Architectures

In streaming systems with a discrete CPU-GPU architecture, leveraging the strengths of both processing units can significantly improve query performance. Existing studies assign entire queries to either the CPU or GPU to reduce data transfer overhead and boost throughput. However, this coarse-grained approach can limit performance for two main reasons. Firstly, PCIe transfer overhead is minimal for small data sizes, and the device preference of each operator within a query may change with variations in data size. Secondly, it neglects performance fluctuations based on the placement location of consecutive operators within the device. To address these issues, we propose dSTREAM, a distributed stream processing system that dynamically maps queries at the operator level on discrete CPU-GPU architectures. dSTREAM adapts to runtime conditions by selecting the optimal device for each operator dynamically. Through dynamic operator-level query mapping without prior knowledge, dSTREAM consistently achieves high performance. Extensive evaluation has confirmed that dSTREAM enhances average throughput by up to 45% and reduces average latency by up to 42.5% across various types of stream SQL queries, regardless of traffic types.