Not-So-Pretty: Studying and Segmenting Multiline Console Logs
Console logs play a crucial role in the development, maintenance, and operation of software systems. However, their diverse and human-oriented formatting presents significant challenges for automated analysis. Most existing log analysis benchmarks and the tools built upon them operate under the assumption that individual log messages correspond to single lines. In reality, console logs frequently span multiple lines and exhibit a wide range of structures and formats, complicating parsing and downstream analysis. In this article, we conduct a qualitative study on a diverse dataset of console logs, identifying 4 structural types and 10 common formats in multiline log segments. We find that although only 3% of the log messages are multiline, they account for 66% of the content by line count. To address the challenge of parsing log messages that span multiple lines, we also propose a deep learning-based preprocessing approach that splits logs into structurally and semantically consistent segments and assigns labels about their format. Our model, trained on a manually labelled dataset, achieves an F1 score of 0.88 for log segmentation and 0.84 for log format classification. We release our dataset and model to support further research in log analysis.
Authors
- Jianchen Zhao (ORCID: https://orcid.org/0000-0002-6362-0106)
- Kundi Yao (ORCID: https://orcid.org/0000-0002-3756-4673)
- Yasutaka Kamei (ORCID: https://orcid.org/0000-0002-7058-1045)
- Weiyi Shang (ORCID: https://orcid.org/0000-0001-6222-7444)
- Masanari Kondo (ORCID: https://orcid.org/0000-0002-6317-7001)
- Hetong Dai (ORCID: https://orcid.org/0000-0001-8465-3380)
- Weiyi Shang
- Yasutaka Kamei
Institutions
- Kyushu University (JP)
- University of Waterloo (CA)
Publication Details
- Journal
- ACM Transactions on Software Engineering and Methodology
- Published
- 2026-11-18
- DOI
- https://doi.org/10.1145/3803019
- Primary Topic
- Software System Performance and Reliability
- Type
- article
- Field-Weighted Citation Impact
- 0.00