Threshold Selection and Attack Type Generalization in XGBoost and Denoising Autoencoder Intrusion Detection
Hybrid intrusion detectors must divide a false-positive budget between supervised and anomaly scores. We study allocation stability, changed alerts, and attack-type transfer using extreme gradient boosting (XGBoost) and a benign-trained denoising autoencoder (DAE). Experiments include cross-day evaluation, fixed-model resampling, type and grouped-family holdouts, matched baselines, and corrected CICIDS2017 and UNSW-NB15 replications. At the 1% calibration target, exploratory Friday recall is 67.46% for DAE alone versus 47.47% for selected fusion, with observed false-positive rates of 7.29% and 5.69%. With re-estimated thresholds, original CICIDS2017 allocations change in 25.0–54.0% of development resamples at the 0.1% target and 65.0–78.5% at 1%; mean retrospective Friday prediction disagreement at 1% is only 0.81–1.04%. Selected fusion gains 5.86 percentage points of macro recall over XGBoost in six individual-type holdouts at 1%. With all four denial-of-service (DoS) types excluded together, however, 1% within-family recall is 19.26% for selected fusion, 14.25% for XGBoost, and 54.78% for DAE; both-Patator exclusion gives 5.46% for selected fusion versus 7.66% for XGBoost. Ordinary-autoencoder fusion performs comparably to DAE fusion in the individual-type protocol, and longer denoising training does not consistently improve detection. On corrected CICIDS2017, selected fusion reaches 66.18% macro recall versus 70.71% for XGBoost. UNSW-NB15 holdouts disable the anomaly branch throughout, with a 4.38% mean test false-positive rate at the 1% target; a separate all-known audit finds rare switching. The contribution separates allocation variability, changed decisions, and budget transfer. Neither stable selection nor gains on individual attack types establish reliable unseen-family detection.
Authors
- Young Im Cho (ORCID: https://orcid.org/0000-0003-0184-7599)
- Zhansaya Abildaeva (ORCID: https://orcid.org/0000-0002-2637-0443)
- Galymzhan Kazhymurat
- Tulegenova Elmira
- Nazerke Karim
- Akmaral Almenova
- Aigul Dauitbayeva
- Nurbek Konyrbayev
Institutions
- Gachon University (KR)
- Korkyt Ata Kyzylorda State University (KZ)
Publication Details
- Journal
- Information
- Published
- 2026-11-16
- DOI
- https://doi.org/10.3390/info17111155
- Primary Topic
- Network Security and Intrusion Detection
- Type
- article
- Field-Weighted Citation Impact
- 0.00