Comparison of Selected Machine Learning Techniques in Cyberattack Anomaly Detection

The digital age has ushered in unprecedented connectivity and technological advancement, which have also introduced a surge in sophisticated and frequent cyber threats. To safeguard systems, anomaly detection has become a cornerstone of cybersecurity, enabling the identification of deviations from normal system behaviour. This study presents a comparative analysis of three machine learning techniques—Isolation Forest, Long Short-Term Memory (LSTM), and Q-Learning—for cyberattack anomaly detection. The study designed and implemented a system using the CICIDS-2017 dataset (2,830,743 records) in Python, preceded by data preprocessing and feature engineering. Evaluation metrics, including Accuracy, F1-Score, and error rates (FPR, FNR) revealed a clear performance hierarchy. The LSTM model proved superior, achieving a near-perfect Accuracy of 99.53% with minimal errors (FPR: 0.35%, FNR: 0.50%). Q-Learning showed strong, adaptive potential, recording an Accuracy of 92.80% and an F1-Score of 90.25%, though with higher error rates (FPR: 8.58%). Conversely, the unsupervised Isolation Forest was inadequate for this labeled task, with metrics around 50%. The findings establish LSTM as ideal for maximum accuracy, Q-Learning as a viable option for dynamic environments, and highlight the limitations of simple unsupervised methods on complex security datasets.