LogERT:Stable log template mining method based on evolving re-search trees

Log data plays an important role in automated operation and maintenance as fine-grained information for recording system events and monitoring system status. Log parsing is a vital task before the automated analysis of log data. Traditional log parsing methods rely on domain knowledge to the extent that it is difficult to generalize across different types of log data. Existing methods based on deep learning are often limited by the feature representation of training samples, struggling to learn the feature structure of a few samples in the training dataset, which leads to insufficient robustness. Moreover, recently proposed parsers based on large models suffer from the instability of the network and the interpretability issues of large models, resulting in poor stability. This paper introduces a log template mining method named LogERT. It uses a search tree with variable depth to generate keywords and initial log templates, and then constructs a search tree for keyword features, eliminating the influence of feature encoding in the search tree on log template generation through backtracking, thus constructing the final log templates. The present method has been extensively experimented on 16 public log datasets. The results show that logERT improves group accuracy (GA) by 4.8% compared to the current advanced log parser group, while its standard deviation decreases by 4.29%, demonstrating that logERT not only has high parsing accuracy but also better stability across datasets with different feature distributions. • Generate initial template using variable depth search tree based on log features. • Reintegrate the initial template using a search tree based on log feature keywords. • Evolutionary search trees for exploring and encoding parsing rules. • Keyword search tree is used to search and update approximate templates.