Anomaly Identification in Cyber Logs Through Machine Learning Techniques

With the rising volume and sophistication of cyber attacks, traditional rule-based security systems have become insufficient to detect emerging and unknown threats. Machine Learning (ML) provides a dynamic and scalable solution for anomaly detection in cyber security logs, enabling systems to identify unusual patterns with minimal human intervention. This paper presents a comprehensive framework for anomaly detection using ML, incorporating supervised, unsupervised, and semi-supervised models. Using the HDFS log dataset, we implement and evaluate Isolation Forest, Auto encoder, and Random Forest algorithms. The results demonstrate that ML techniques significantly enhance detection accuracy and reduce false positives, with Random Forest achieving the highest performance across all evaluation metrics.