Advancing Network Threat Detection Through Standardized Feature Extraction and Dynamic Ensemble Learning

This research proposes an innovative approach to network intrusion detection by employing an ensemble of machine learning models to identify suspicious network traffic. Building on previous studies, this research focuses on three key components: developing a standardized feature extraction framework, selecting and training a set of machine learning models, and designing a novel ensemble classification algorithm that leverages their respective strengths. The standardized feature extraction framework emphasizes metadata and flow-level statistics, with the goal of reducing bias and improving generalization across diverse network environments. An ensemble of machine learning models (Random Forest, Isolation Forest, Gaussian Mixture Models, Quadratic Discriminant Analysis, AdaBoost, XGBoost, Convolutional Neural Networks, and Recurrent Neural Networks) are evaluated with the goal of leveraging their unique strengths in detecting various types of network anomalies. A novel ensemble classification algorithm, Ford Class Specific Weighted Values (Ford-CSWV), is proposed to weight model outputs based on class-specific validation performance, with the goal of enhancing the ensemble's overall detection capability. Experimental results demonstrate that the ensemble classifier achieved an accuracy of 97.92%, with balanced precision and recall for both benign and malicious traffic. While gains over individual classifiers were minimal, the ensemble provides stable performance across different traffic types, reducing the risk of false positives. This work provides a blueprint for developing more robust and adaptable network intrusion detection systems.