A First-Contact Evaluation of logcrux: Deterministic Local Log Analysis for Failure Detection
Log analysis tools are usually described by how many formats they parse and how accurately they classify an evaluation set drawn from the same data as the training set. Neither number shows whether the tool raises an alarm when a system fails. We study that question for logcrux, an open-source (Apache-2.0) command-line log triage tool for offline and air-gapped environments that runs fully on the local machine and decides deterministically. We build a corpus of 1,941 log captures from running systems in which we induced faults: 54 cloud virtual machines covering 19 releases of 8 Linux distributions on two architectures, a Kubernetes cluster, official container images of common server software, and a backup lab. Each capture was labelled before scoring with the verdict an operator should have received. Each capture is scored on the verdict the tool reports, and a fault reported as clean is counted as the primary error. We report three results. First, recognising a format did not imply detecting its failures. A release that raised parser correctness from 55% to 80% changed no verdict. Faults that appear as one log event, such as a killed database backend or a failed backup, were missed on all 12 operating systems tested, while faults that produce many lines were caught on all 12. Second, a first-contact protocol, which scores each new round of unseen software before any fix is written, gives 42 of 77 faults detected (55%, 95% CI 43–65%) against 1,563 of 1,582 (98.8%) on the development set. Third, we compare the tool with a plain search for failure words run on the same captures. On the full corpus the search detects 92.5% of faults but alarms on 31.8% of healthy captures, against 98.8% and 0% for the tool on its development set. On the 64 held-out faults from the container and Kubernetes rounds, the search detects 86% against the tool's 53% at first contact, at the cost of alarms on 6 of 27 healthy captures. The tool's misses on unseen software came both from incident types no detector covered and from failure phrases it had not seen, and both kinds kept occurring as rounds accumulated. We release the corpus, the fault-induction scripts and the scorer. LinksWebsite and documentation: https://logcrux.comSource code: https://github.com/ravipatip/logcruxPyPI package: https://pypi.org/project/logcrux/Companion dataset: https://doi.org/10.5281/zenodo.24427636 Use of AI tools. The authors conceived the study, designed the first-contact evaluation methodology and fault scenarios, provisioned and managed the cloud servers and container testbeds, and reviewed and verified all code, labels, and results. Claude (Anthropic) and a locally run Qwen 3.8 27B (Alibaba) were used as general coding and language-editing assistants. All reported results come from real systems and are reproducible from the released captures and scorer. The authors take full responsibility for the work.
Authors
- Bharadwaj Ravipati (ORCID: https://orcid.org/0009-0003-9777-6998)
- Prameela Ravipati (ORCID: https://orcid.org/0009-0006-4481-6679)
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-11-21
- DOI
- https://doi.org/10.5281/zenodo.24427594
- Primary Topic
- Software System Performance and Reliability
- Type
- preprint