Modified Counter: A Fast and Dynamic Structure for Locating High-Frequency Items in Data Streams

The demand for high-speed data stream processing has increased as networks have become more complex. Specific challenges include managing large-scale streams and addressing issues caused by data skewness and the presence of anomalies. As such, this paper introduces Modified Counter (MC), a novel data structure in which conventional XOR operations are replaced by a unique one-item lookup process wherein each item hash is partitioned into a quotient for bin selection and a remainder for precise entry location. This technique significantly reduces hash collisions, thereby improving retrieval accuracy and fingerprint indexing times, which increases both processing speed and spatial efficiency. Counter structures and entry sorting are further optimized by selectively evicting low-frequency items and prioritizing high-frequency data, which is beneficial for skewed distributions and high-conflict scenarios. As a result, MC offers unprecedented load balancing and adaptability, particularly for high-speed data streams. Validation experiments showed that MC outperformed conventional structures, consistently achieving the highest throughput for frequency queries, the lowest error rates, and the highest precision for heavy hitter queries, which are critical for high-speed data processing. Notably, an insertion rate of 19.4 Mbps was achieved with the IP dataset. The average absolute error was approximately 10<sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">-1.5</sup> with a memory of only 0.1 MB, improving to below 10<sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">-2.5</sup> with 1.0 MB, representing an improvement of ~2 orders of magnitude over conventional models. These results suggest MC to be a valuable structure for reducing memory dependency by more efficiently locating high-frequency items in high-speed data streams.