Certainly Uncertain: Demystifying ML Uncertainty for Active Learning in Network Monitoring Tasks

Artificial Intelligence (AI), particularly Machine Learning (ML), has become prominent in network monitoring, yet its practical adoption, such as for anomaly and intrusion detection, remains limited. Standard AI/ML methods often exclude experts, reducing trust and hindering practical implementations. Active Learning (AL) allows to integrate admins and their expert knowledge into the ML loop by leveraging expert-labeled data. Together with self-training and automated decisions, AL can enhance model performance, trust, and the ability to adapt to system changes. In this work, we evaluate uncertainty-based AL in network monitoring, offering a comprehensive parameter study for best practices in real-world AI/ML adoption. To this end, we evaluate stream-based and pool-based AL across four datasets for various monitoring use cases and conduct a parameter study on ten uncertainty measures, thereby identifying scenarios benefiting from self-training. By analyzing the impact of admin competence on model performance, we offer actionable guidelines towards the practical implementation of AL.