Rigorous Evaluation of Machine Learning-Based Intrusion Detection Against Adversarial Attacks

The rapid growth of the Internet of Things (IoT) has engendered profound security challenges. Intrusion detection system (IDS) is a security measure to mitigate these challenges by continuously monitoring system data and alerting to any suspicious activity. While machine learning (ML) has emerged as a promising IDS solution, its vulnerability to adversarial attacks raises concerns about the reliability of these systems. In this paper, we present a rigorous evaluation framework to assess the performance of ML-based IDS against various adversarial attacks in IoT environments. Our framework employs a wide range of adversarial attack techniques, including white-box, gray-box, and black-box adversarial attacks, across four realistic and recent IoT intrusion datasets. Our results showed that the intrusion detection performance of state-of-the-art ML and DL models deteriorates by up to 49.5 × under adversarial attacks. This observation indicates an urgent need for more resilient ML-IDS solutions against adversarial attacks in IoT systems.