Large Scale Cyber Security Log Classification Using Semi-Supervised Clustering

In this paper we present a semi-supervised approach developed with the aim addressing the challenge of large-scale cyber security log entry classification that is faced by organizations that lack significant in-house expertise. Our approach is to combine Hierarchical Agglomerative Clustering with two well established unsupervised clustering approaches DBSCAN and K-Medoid to classify the Curtin Dataset. We describe in detail the results obtained using a range of experimental setups and discuss the practicality of the proposed method. In addition to the results, we provide a new large-scale cyber security dataset that can be used in cyber security related research.