Eleven Detectors on Eleven Corpora: A Seeded Cross-dataset Benchmark for IoT and IIoT Intrusion Detection

comparisons of intrusion detectors for the Internet of Things are hard to read against one another: Each fixes its own preprocessing, its own split, its own metric, and-most consequentially-its own single run.We report a benchmark built to remove those degrees of freedom.Eleven detectors, spanning linear models, three tree ensembles and seven neural architectures, are driven through one harness on eleven public IoT and IIoT corpora with an identical pipeline and five seeds per cell: 605 runs, every one of them recorded.The ordering is unambiguous and it is not the one the recent literature would predict.The three tree ensembles hold the first three ranks on all eleven corpora; the best tree beats the best neural model on all ten corpora that still discriminate and ties it on the saturated eleventh, by a mean margin of 0.104 macro-F1 (bootstrap 95% CI 0.065-0.141,conditional on these eleven corpora).A Friedman test rejects rank equality (χ2 = 86.4,p = 2.7×10-14) and a pre-specified tree-versus-neural contrast is significant both at the family level (Wilcoxon p = 0.001) and for each of the 21 tree-neural pairs after Holm correction.The ordering is unchanged when LightGBM, the two corpora affected by a row-cap defect, or IoTNet24, the one corpus without a peer-reviewed description, is removed from the grid.Rerun under one class-weighting policy for every detector, on five independent partitions per corpus with the row cap enforced per corpus, the best tree still beats the best neural model on all ten corpora that completed.On the one partition whose test part the tuning never saw, budget-matched tuning narrows the gap to 0.059 macro-F1 (95% CI 0.021-0.100),about half its untuned size on the same partition, and to under 0.025 on five of the ten.The gap is not bought with compute: XGBoost fits 34× faster and scores 21× faster than the strongest neural model, and only two far less accurate detectors score a flow faster: the MLP (0.21 µs) and logistic regression (0.24 µs, against 1.02 µs for XGBoost).Three further results concern what a benchmark of this kind can and cannot establish.Two of the eleven corpora no longer discriminate-on N-BaIoT all eleven detectors, logistic regression included land within 0.003 of a perfect score.Run to-run instability is a property of the (detector, corpus) pair and not of the model family: sixteen of the 121 cells move by more than 0.05 macro-F1 across seeds, and the widest belongs to a gradient-boosted ensemble returning 0.897 on four seeds and 0.543 on the fifth.In the widest of these cells the five runs fall into two groups with an empty interval between them, so the customary mean ± s.d.describes no run that occurred, while accuracy cannot see them at all: one cell moves 0.205 in macro-F1 while its accuracy moves 0.0005.We close with five preprocessing and harness defects, four silent end to end, each of which produced a filled results table that was arithmetically meaningless-and with the released harness that reproduces every number here from one command.The results concern detectors applied to single-flow features; they are a within-dataset benchmark repeated over eleven corpora, not a study of transfer between them.

Authors

Institutions

Publication Details

Journal
International journal of intelligent engineering and systems
Published
2026-11-17
DOI
https://doi.org/10.22266/ijies2026.1231.70
Primary Topic
Network Security and Intrusion Detection
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Eleven Detectors on Eleven Corpora: A Seeded Cross-dataset Benchmark for IoT and IIoT Intrusion Detection

Dhahir Abdulhade Abdulah, Layla A. Al Hak
3 links
International journal of intelligent engineering and systems
Network Security and Intrusion Detection
article

Eleven Detectors on Eleven Corpora: A Seeded Cross-dataset Benchmark for IoT and IIoT Intrusion Detection

Dhahir Abdulhade Abdulah, Layla A. Al Hak
article en

Abstract

comparisons of intrusion detectors for the Internet of Things are hard to read against one another: Each fixes its own preprocessing, its own split, its own metric, and-most consequentially-its own single run.We report a benchmark built to remove those degrees of freedom.Eleven detectors, spanning linear models, three tree ensembles and seven neural architectures, are driven through one harness on eleven public IoT and IIoT corpora with an identical pipeline and five seeds per cell: 605 runs, every one of them recorded.The ordering is unambiguous and it is not the one the recent literature would predict.The three tree ensembles hold the first three ranks on all eleven corpora; the best tree beats the best neural model on all ten corpora that still discriminate and ties it on the saturated eleventh, by a mean margin of 0.104 macro-F1 (bootstrap 95% CI 0.065-0.141,conditional on these eleven corpora).A Friedman test rejects rank equality (χ2 = 86.4,p = 2.7×10-14) and a pre-specified tree-versus-neural contrast is significant both at the family level (Wilcoxon p = 0.001) and for each of the 21 tree-neural pairs after Holm correction.The ordering is unchanged when LightGBM, the two corpora affected by a row-cap defect, or IoTNet24, the one corpus without a peer-reviewed description, is removed from the grid.Rerun under one class-weighting policy for every detector, on five independent partitions per corpus with the row cap enforced per corpus, the best tree still beats the best neural model on all ten corpora that completed.On the one partition whose test part the tuning never saw, budget-matched tuning narrows the gap to 0.059 macro-F1 (95% CI 0.021-0.100),about half its untuned size on the same partition, and to under 0.025 on five of the ten.The gap is not bought with compute: XGBoost fits 34× faster and scores 21× faster than the strongest neural model, and only two far less accurate detectors score a flow faster: the MLP (0.21 µs) and logistic regression (0.24 µs, against 1.02 µs for XGBoost).Three further results concern what a benchmark of this kind can and cannot establish.Two of the eleven corpora no longer discriminate-on N-BaIoT all eleven detectors, logistic regression included land within 0.003 of a perfect score.Run to-run instability is a property of the (detector, corpus) pair and not of the model family: sixteen of the 121 cells move by more than 0.05 macro-F1 across seeds, and the widest belongs to a gradient-boosted ensemble returning 0.897 on four seeds and 0.543 on the fifth.In the widest of these cells the five runs fall into two groups with an empty interval between them, so the customary mean ± s.d.describes no run that occurred, while accuracy cannot see them at all: one cell moves 0.205 in macro-F1 while its accuracy moves 0.0005.We close with five preprocessing and harness defects, four silent end to end, each of which produced a filled results table that was arithmetically meaningless-and with the released harness that reproduces every number here from one command.The results concern detectors applied to single-flow features; they are a within-dataset benchmark repeated over eleven corpora, not a study of transfer between them.

International journal of intelligent engineering and systemsVol. 19(12)
University of Diyala (IQ)
Network Security and Intrusion Detection
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon