Enhancing Cyber Threat Detection Accuracy: An AI-Powered Approach with Feature Selection and Machine Learning with Ensemble Learning for Cyber Threat Detection

The rapid evolution of cyber threats necessitates advanced detection mechanisms to ensure robust network security. This study presents an AI-driven ensemble-based cyber threat detection system leveraging the CICIDS2017 dataset. Our multi-stage methodology integrates data preprocessing, attack data filtering, feature selection, and machine learning model evaluation. Data preprocessing involves cleaning, normalization, and handling missing values to enhance data quality. Attack data filtering isolates malicious and benign traffic for effective model training. Feature selection employs the Random Forest Regressor to identify key predictive attributes. The proposed system evaluates multiple machine learning algorithms, including Naive Bayes, Quadratic Discriminant Analysis (QDA), and Multi-Layer Perceptron (MLP), considering accuracy, precision, and computational efficiency. Furthermore, an ensemble model aggregates predictions from multiple classifiers to enhance detection reliability. A web-based Streamlit application facilitates real-time attack classification, presenting ensemble-based probabilistic predictions for eight attack types, including DDoS, DoS variants, and infiltration attempts. The results highlight the potential of integrating ensemble learning with feature selection and preprocessing techniques to reduce false positives, improve detection accuracy, and enable real-time threat mitigation in large-scale networks.