An Accurate Flow Size Measurement Framework Based on Machine Learning

Per-flow size measurement plays a crucial role in high-speed networks, providing essential data support for key functions such as routing, load balancing, and security. However, the scarcity of high-speed on-chip cache memory forces existing approaches to employ shared resource strategies for massive network streams. This results in steady hash noise that is difficult to eliminate, ultimately reducing estimation accuracy. To alleviate the impact of this issue, this paper proposes Separate-Learning Sketch (SLS), a machine learning-based per-flow size measurement framework that integrates real-time on-chip traffic capture with offline learning-based estimation. By leveraging such a hybrid design, SLS effectively mitigates hash noise and enhances the estimation accuracy of per-flow size. First, we propose a novel Sketch capable of distinguishing flow sizes, capturing flows that exceed a predefined threshold in real-time, and sampling them into an off-chip hash table for subsequent measurement and analysis. On the other hand, the offline measurement module records the size of each flow and employs a sampling module to sample overflow flows to off-chip memory for machine-learning model training. The off-chip hash table stores the sampled flow sizes, which serve as labels for training models tailored to different flow distributions. After each measurement, the Sketch is transmitted to the off-chip, where the trained model is leveraged to predict the overflowed flows’ sizes accurately. Experimental results on real-world network traffic datasets demonstrate that, compared to the representative Elastic Sketch, SLS achieves a 60.55% reduction in ARE within a compact 100KB memory budget. Additionally, compared to LCMS, SLS reduces ARE by 52.75%, highlighting its superior measurement accuracy.