Encoding Domain Knowledge in Log Analysis

Software developers often use logs to, e.g., investigate bugs, familiarize themselves with the underlying system, or improve performance. To do so, they commonly rely on text editors or their own scripts. This lack of appropriate tooling remains a primary challenge in the industrial application of log analysis (state-of-the-practice), despite many tools and techniques proposed by previous scientific studies (state-of-the-art). To aid in bridging this gap between industry and academia, between state-of-the-practice and state-of-the-art, we zoom in on the ways developers perform log analysis. In particular, we conduct an exploratory case study to understand what structures developers identify in logs and how they utilize their knowledge in this process. Based on the results of the case study, we identify two classes of features, one related to encoding domain knowledge and another one related to sharing domain knowledge. We implement two features from the first class in an open-source log analysis platform designed in collaboration with our industrial partner. To evaluate the impact of the implemented features on log analysis, we conduct a user evaluation with software developers from our industrial partner. During this evaluation developers complete several tasks using the features and complete a usability questionnaire. Results show that users are able to encode their domain knowledge about the logs during their analysis. Furthermore, we observe that participants value highly ease of use and indicate an interest in using the features in their current practice. This sentiment is reflected in the resulting scores of the usability questionnaire, indicating above-average usability. Our findings pave the way to bridge the gap between academia and industry and facilitate the application of advanced log analysis approaches in industry.