Anomaly Detection with Windows Event Logs: A comparative study between traditional and ML based approaches

The increasing number of cyber threats raises the need for better detection systems. Anomaly detection based on log data can help to create more reliable and secure computer systems. Log analysis tools, such as Splunk, can help to analyze the underlying data and therefore support the creation of techniques to counteract cyber attacks. This thesis aims to compare a traditional rule-based approach to several Machine Learning (ML) models and evaluate their strengths and weaknesses in detecting attacks based on Windows Event Log (WEL)s. The used ML classifiers are Random Forest, Multilayer Perceptron (MLP), Support Vector Machine (SVM), Logistic Regression, Naive Bayes, and One-Class Support Vector Machine (OCSVM). These classifiers were used to evaluate their performance compared to the traditional rule-based anomaly detection approach, considering four different attack scenarios: Brute Force, Bypass User Account Control, Remote Services, and Disable Windows Event Logging. Overall, the results showed that the Random Forest classifier proved to be the most effective ML model, often outperforming the traditional However, if only a small amount of attack data is available, the traditional approach can outperform ML ones. Future work should include more attacks, tactics, and techniques to allow for a more all-encompassing view of the differences between the traditional and modern approaches. Moreover, since attack data is limited, generating new attack samples could improve the robustness and generalization of detection. This would then allow to create better Intrusion Detection System (IDS) in the future.