LightGBM-Based Anomaly Detection System for Modern Network Traffic Using UNSW-NB15 Dataset

Intrusion detection systems play a vital role in identifying unauthorized activities and safeguarding against emerging threats. This study highlights the capabilities of the UNSW-NB15 dataset, a modern benchmark for network anomaly detection, capturing diverse network traffic patterns. Unlike earlier research that often lacked uniform validation or relied on limited evaluation methods, we leverage the LightGBM classifier for binary anomaly detection, integrating advanced feature engineering, preprocessing, and selection techniques. Our approach was evaluated using various experimental configura- tions, including ten-fold cross-validation on training, testing, and combined datasets, yielding F1 scores of 97.21%, 98.33%, and 96.21%, respectively. Furthermore, when trained exclusively on the training data, our model achieved a 92.96% F1 score on the independent test dataset, demonstrating strong generalization to unseen data. Comparative analysis reveals that our method surpasses prior models across multiple metrics, underscoring its effectiveness in identifying anomalies within contemporary network environments.