Machine Learning-Based Cyberattack Detection in NFV: Performance, Efficiency and Explainability Analysis

Network Function Virtualization (NFV) is a novel concept in computer networking that enhances scalability and flexibility in network services. Its increasing complexity and highly dynamic nature challenge reliable and efficient cyberattack detection. This study evaluates the predictive performance and computational efficiency of ensemble machine learning (ML) models, including Random Forest, eXtreme Gradient Boosting (XGBoost), and CatBoost, using three NFV-specific subsets of the VNFCYBERDATA dataset, and compares them with a stacking-based meta learner for classifying benign and malicious network traffic. To enhance the ML workflow, a reproducible preprocessing pipeline and automated hyperparameter optimization with the Optuna framework were employed. Performance was assessed using: precision, recall, F1-score, false positive rate (FPR), Precision–Recall Area Under the Curve (PR-AUC), training and inference speed, memory usage, and serialized model size. SHapley Additive exPlanations (SHAP) was used to interpret model predictions and identify the strongest indicators of cyberattacks. The results demonstrate that Random Forest and the stacked meta learner consistently achieved the highest predictive performance, attaining precision and PR-AUC values of up to 99%, while retaining very low false positive rates. Meanwhile, XGBoost and CatBoost demonstrated higher efficiency, resulting in substantially faster training and inference times (10–12 ms), which makes them better suited for latency-sensitive deployments. These findings support the applicability of boosting models in resource-constrained environments, while bagging and stacking models provide robust detection in high-dimensional and imbalanced datasets. SHAP analysis identified traffic volume, transport-layer protocol flags, and network flow characteristics as the strongest indicators of malicious activity in NFV networks, providing deeper insights into model predictions.