On detection latencies of network intrusion detectors – discussion and application

Abstract The ever-evolving landscape of attacks and the growing complexity of ICT systems make crafting anomaly-based intrusion and error detectors difficult: they must accurately detect attacks and promptly perform detections. Although improving and comparing the detection capability is the focus of most research works, the timeliness of the detection is less considered and often insufficiently evaluated or discussed. In this paper, we argue the relevance of measuring the temporal latency of attacks, and we propose an evaluation approach for detectors to ensure a trade-off between correct and in-time detection. Briefly, the approach relates the false positive rate to the temporal latency of attacks, ultimately leading to guidelines for configuring a detector. We discuss and apply the strategy to compose datasets for intrusion detection that can support the computation of our metrics. We exercise our approach by evaluating different intrusion and error detectors in three industrial cases: i) an embedded railway on-board system that optimizes public mobility, ii) an edge device for the Industrial Internet of Things, and iii) an IoT network that monitors an industrial facility. Results show that considering latency in addition to traditional metrics like the false positive rate, precision, and recall gives an additional fundamental perspective on the actual performance of the detector and should be considered when assessing and configuring intrusion detectors.