System Logs Anomaly Detection. Are we on the right path?

System logs are universally used for monitoring user access, performance, and behavior in software applications. Large-scale industrial systems generate an immense volume of logs, which are difficult to handle with human capabilities. Therefore, an automated method is essential for filtering vast amounts of data. System log anomaly detection is crucial in the security field for identifying system failures, sophisticated internal attacks, and other deviations from the norm. This research area requires further development, as most Deep Learning solutions in the literature are semi-supervised. This poses a significant limitation since these solutions are impractical for large-scale ecosystems due to the high cost of labeling data. This paper introduces a method that replaces the supervised phase of semi-supervised methods with fully unsupervised heuristics, utilizing the elbow method, interquartile range, and Simulated Annealing. The unsupervised results are comparable to the semi-supervised State of the Art while demonstrating greater applicability in real-world applications. This work proposes a more suitable benchmark for the log anomaly outlier detection problem, where the training data include both normal and abnormal sequences and precede the test sessions in time. Additionally, it presents metrics on distinct log sequences to mitigate the impact of unbalanced anomaly types.