Protocol-Constrained Adversarial Evaluation of MQTT-Based Intrusion Detection Systems: Quantifying the Realism Gap
Abstract: Machine Learning Intrusion Detection Systems (ML IDS) are vulnerable to adversarial attacks—intentional perturbations that cause misclassification. Most studies generate unconstrained feature-space attacks that ignore network-protocol rules, creating a realism gap in resource-constrained Internet of Things (IoT) networks where high theoretical evasion rates may not reflect a genuinely deployable threat. Following a Design Science Research methodology, this study develops and empirically evaluates a protocol-aware adversarial evaluation framework for Message Queuing Telemetry Transport (MQTT)-based IDS. A Random Forest baseline classifier was trained on the publicly available MQTTEEB-D dataset (n = 222,813). A three-layer protocol constraint model (field, temporal, and state-machine rules) was formalized, with field and temporal constraints encoded directly into the fitness objective of an evolutionary attack generator and benchmarked against two unconstrained feature-space attacks—Fast Gradient Sign Method (FGSM) and Projected Gradient Descent (PGD)—using Evasion Success Rate (ESR) and Protocol Validity Rate (PVR). The baseline classifier achieved 91% accuracy and a macro F1 score of 0.86. While PGD reached 24.16% ESR, it yielded 0% PVR; in contrast, the protocol-aware evolutionary attack achieved 6.00% ESR while maintaining 71.3% PVR (McNemar's p = 8.30 × 10⁻¹⁷, Cohen's h = 0.533). These findings demonstrate that traditional feature-space attacks overestimate deployable risk, whereas protocol-compliant attacks reveal a smaller yet statistically significant real-world vulnerability. Keywords: Adversarial Machine Learning; Intrusion Detection Systems; MQTT Protocol; Internet of Things Security; Protocol-Aware Perturbation
Authors
- Elisha Odira Abade (ORCID: https://orcid.org/0000-0002-8978-4213)
- Benjamin Isaac Macharia Kinyanjui
Institutions
- University of Nairobi (KE)
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-11-17
- DOI
- https://doi.org/10.5281/zenodo.24311252
- Primary Topic
- Adversarial Robustness in Machine Learning
- Type
- preprint