Now You See Me / Now You Don’t: Constrained Adversarial Attacks in Network Intrusion Detection Across Datasets and Machine Learning Models

As the number of cyber-attacks is rising, Network Intrusion Detection Systems (NIDS) are becoming increasingly important for the detection and identification of malicious traffic, and Machine Learning techniques are gaining traction for the classification of network traffic. However, especially Deep Learning models are vulnerable to adversarial examples— subtle input perturbations that cause misclassification. While adversarial examples for images must be imperceptible, network data perturbations face complex, domain-specific constraints.We examine the vulnerability of NIDS to adversarial attacks, and compare six machine learning architectures and three black-box attack methods on four Network Intrusion Detection (NID) datasets. We define a threat model that provides minimal knowledge and limited access to an adversary, and use a highly restricted feature subset to apply perturbations to. Our results show attack success rates of 0.1-36.9%, with large, dataset-specific differences between architectures. In a second step, we apply three methods to minimize the size of the perturbations and find that the Pointwise attack is especially well suited for NID data. Finally, we analyze perturbed features and find single features greatly impact classification.