Analyzing shallow and deep classifiers in detecting unseen attacks on internet of things network

Intrusion Detection Systems (IDSs) for Internet of Things (IoT) networks face increasing difficulty when encountering attack behaviors that differ from those observed during training. While most existing IDS solutions assume closed set conditions, real deployments often experience distribution shifts caused by evolving or previously unobserved attack patterns. This study evaluates the robustness of widely used machine learning and deep learning classifiers when trained on a subset of attack classes and tested on withheld attack categories within IoT traffic datasets. We conduct a comprehensive experimental study using two publicly available IoT traffic datasets, N_BaIoT and BoT-IoT, evaluating thirteen shallow and deep learning classifiers under both standard multiclass (seen-attack) settings and held-out(unseen) attack-class scenarios. The results shows significant miss rates across most classifiers when evaluated on unseen attack classes. This underlines the limitations of closed-set learning for this task. Among the evaluated models, XGBoost consistently achieves higher overall accuracy and lower false negative rates compared to other classifiers, outperforming the least effective DenseNet-based model by up to 37% in terms of mean accuracy across multiple experimental setups. However, despite achieving high classification accuracy under distribution shifts, these models continue to assign unseen attack samples to known classes, underscoring their inability to explicitly identify traffic as novel. The findings emphasize that strong closed-set performance does not necessarily equate to true unseen attack detection, motivating the need for open-set and novelty-aware intrusion detection approaches in realistic IoT security scenarios.