Comparative Evaluation of Machine Learning Models for IoT Intrusion Detection Under Class Imbalance and Feature Drift

Abstract Intrusion-detection studies for Internet of Things (IoT) systems commonly report strong classification performance, but comparisons are often difficult because class imbalance, probability calibration, feature count, and distribution shift are evaluated separately. This study performs a controlled empirical comparison of five conventional machine-learning configurations under a common eight-class security benchmark. The benchmark contains 18,000 development observations with 28 traffic-like features and an imbalanced class distribution, while a separate 5,000-observation shifted set introduces feature scaling, offsets, and additive noise. Logistic regression, random forest, Extra Trees, histogram gradient boosting, and calibrated Extra Trees are evaluated across three independent random seeds. On the development test split, calibrated Extra Trees achieved balanced accuracy of 0.981 ± 0.005, macro-F1 of 0.984 ± 0.004, and Matthews correlation coefficient of 0.983 ± 0.005. Under feature drift, balanced accuracy remained 0.981 ± 0.006 and macro-F1 0.983 ± 0.005. Probability calibration reduced expected calibration error for Extra Trees from 0.182± 0.006 to 0.014± 0.001 on the development split. A feature-ablation experiment showed that 18 of 28 features retained macro-F1 of 0.978 ± 0.002, whereas reducing the representation to 10 features lowered macro-F1 to 0.931±0.012. The results provide a reproducible comparison of standard classifiers and illustrate the practical value of reporting balanced metrics, calibration, and feature-drift performance together. No new classifier or intrusion-detection architecture is proposed.

Authors

Publication Details

Journal
Research Square
Published
2026-11-17
DOI
https://doi.org/10.21203/rs.3.rs-11738586/v1
Primary Topic
Network Security and Intrusion Detection
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Comparative Evaluation of Machine Learning Models for IoT Intrusion Detection Under Class Imbalance and Feature Drift

Lucien beyli
3 links
Research Square
Network Security and Intrusion Detection
preprint

Comparative Evaluation of Machine Learning Models for IoT Intrusion Detection Under Class Imbalance and Feature Drift

Lucien beyli
preprint en

Abstract

Abstract Intrusion-detection studies for Internet of Things (IoT) systems commonly report strong classification performance, but comparisons are often difficult because class imbalance, probability calibration, feature count, and distribution shift are evaluated separately. This study performs a controlled empirical comparison of five conventional machine-learning configurations under a common eight-class security benchmark. The benchmark contains 18,000 development observations with 28 traffic-like features and an imbalanced class distribution, while a separate 5,000-observation shifted set introduces feature scaling, offsets, and additive noise. Logistic regression, random forest, Extra Trees, histogram gradient boosting, and calibrated Extra Trees are evaluated across three independent random seeds. On the development test split, calibrated Extra Trees achieved balanced accuracy of 0.981 ± 0.005, macro-F1 of 0.984 ± 0.004, and Matthews correlation coefficient of 0.983 ± 0.005. Under feature drift, balanced accuracy remained 0.981 ± 0.006 and macro-F1 0.983 ± 0.005. Probability calibration reduced expected calibration error for Extra Trees from 0.182± 0.006 to 0.014± 0.001 on the development split. A feature-ablation experiment showed that 18 of 28 features retained macro-F1 of 0.978 ± 0.002, whereas reducing the representation to 10 features lowered macro-F1 to 0.931±0.012. The results provide a reproducible comparison of standard classifiers and illustrate the practical value of reporting balanced metrics, calibration, and feature-drift performance together. No new classifier or intrusion-detection architecture is proposed.

Research Square
Network Security and Intrusion Detection
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon