QPOPSS: Query and Parallelism Optimized Space-Saving for finding frequent stream elements

The frequent elements problem, a key component in demanding stream-data analytics, involves selecting elements whose occurrence exceeds a user-specified threshold. Fast, memory-efficient ϵ -approximate synopsis algorithms select all frequent elements but may overestimate them depending on ϵ (user-defined parameter). Evolving applications demand performance only achievable by parallelization. However, algorithmic guarantees concerning concurrent updates and queries have been overlooked. We propose Query and Parallelism Optimized Space-Saving (QPOPSS ), providing concurrency guarantees. A cornerstone of the design is a new approach for the main data structure for the Space-Saving algorithm, enabling support of very fast queries. QPOPSS integrates this, minimal overlap with concurrent updates, with the distribution of work and fine-grained synchronization among threads, swiftly balancing high throughput, high accuracy, and low memory consumption. Our analysis shows space and approximation bounds under various concurrency and data distribution conditions. Our empirical evaluation relative to representative state-of-the-art methods reveals that QPOPSS 's multi-threaded throughput scales linearly while maintaining the highest accuracy, with orders of magnitude smaller memory footprint. • We propose the memory-conservative QPOPSS, for frequent stream element estimation. • We propose QOSS, a query optimized algorithmic implementation of Space-Saving. • We provide an extensive empirical evaluation and open access to code and data. • Compared to state-of-the-art QPOPSS is highly accurate with small memory footprint. • QPOPSS excels at processing highly skewed data streams with large queries.