Eleven Detectors on Eleven Corpora: A Seeded Cross-dataset Benchmark for IoT and IIoT Intrusion Detection
comparisons of intrusion detectors for the Internet of Things are hard to read against one another: Each fixes its own preprocessing, its own split, its own metric, and-most consequentially-its own single run.We report a benchmark built to remove those degrees of freedom.Eleven detectors, spanning linear models, three tree ensembles and seven neural architectures, are driven through one harness on eleven public IoT and IIoT corpora with an identical pipeline and five seeds per cell: 605 runs, every one of them recorded.The ordering is unambiguous and it is not the one the recent literature would predict.The three tree ensembles hold the first three ranks on all eleven corpora; the best tree beats the best neural model on all ten corpora that still discriminate and ties it on the saturated eleventh, by a mean margin of 0.104 macro-F1 (bootstrap 95% CI 0.065-0.141,conditional on these eleven corpora).A Friedman test rejects rank equality (χ2 = 86.4,p = 2.7×10-14) and a pre-specified tree-versus-neural contrast is significant both at the family level (Wilcoxon p = 0.001) and for each of the 21 tree-neural pairs after Holm correction.The ordering is unchanged when LightGBM, the two corpora affected by a row-cap defect, or IoTNet24, the one corpus without a peer-reviewed description, is removed from the grid.Rerun under one class-weighting policy for every detector, on five independent partitions per corpus with the row cap enforced per corpus, the best tree still beats the best neural model on all ten corpora that completed.On the one partition whose test part the tuning never saw, budget-matched tuning narrows the gap to 0.059 macro-F1 (95% CI 0.021-0.100),about half its untuned size on the same partition, and to under 0.025 on five of the ten.The gap is not bought with compute: XGBoost fits 34× faster and scores 21× faster than the strongest neural model, and only two far less accurate detectors score a flow faster: the MLP (0.21 µs) and logistic regression (0.24 µs, against 1.02 µs for XGBoost).Three further results concern what a benchmark of this kind can and cannot establish.Two of the eleven corpora no longer discriminate-on N-BaIoT all eleven detectors, logistic regression included land within 0.003 of a perfect score.Run to-run instability is a property of the (detector, corpus) pair and not of the model family: sixteen of the 121 cells move by more than 0.05 macro-F1 across seeds, and the widest belongs to a gradient-boosted ensemble returning 0.897 on four seeds and 0.543 on the fifth.In the widest of these cells the five runs fall into two groups with an empty interval between them, so the customary mean ± s.d.describes no run that occurred, while accuracy cannot see them at all: one cell moves 0.205 in macro-F1 while its accuracy moves 0.0005.We close with five preprocessing and harness defects, four silent end to end, each of which produced a filled results table that was arithmetically meaningless-and with the released harness that reproduces every number here from one command.The results concern detectors applied to single-flow features; they are a within-dataset benchmark repeated over eleven corpora, not a study of transfer between them.
Authors
- Dhahir Abdulhade Abdulah (ORCID: https://orcid.org/0009-0008-8726-838X)
- Layla A. Al Hak
Institutions
- University of Diyala (IQ)
Publication Details
- Journal
- International journal of intelligent engineering and systems
- Published
- 2026-11-17
- DOI
- https://doi.org/10.22266/ijies2026.1231.70
- Primary Topic
- Network Security and Intrusion Detection
- Type
- article
- Field-Weighted Citation Impact
- 0.00