Incident Knowledge Graphs for Site Reliability Engineering: Connecting Alerts, Runbooks, Services, Deployments, and Postmortems
On-call engineers responding to a production incident must typically search across several disconnected systems — alerting dashboards, wikis, service catalogs, deployment logs, and postmortem archives — to reconstruct the operational context needed for diagnosis. This fragmentation slows response and causes institutional knowledge captured in past postmortems to go unused in subsequent, related incidents. This article presents Incident Knowledge Graphs (IKG), a framework that represents alerts, runbooks, services, deployments, and postmortems as typed nodes and relations in a unified, continuously updated property graph, and applies graph traversal combined with dense embedding search to retrieve contextually relevant information during live incidents. The framework was evaluated on a benchmark of 640 held-out on-call retrieval queries and piloted over a twelve-month period across a multi-service production environment. The graph-plus-embedding hybrid retrieval approach achieved 0.86 precision at five results and 0.83 mean reciprocal rank, outperforming keyword search, tag-based lookup, and embedding-only retrieval baselines. Field deployment of the IKG was associated with a reduction in median time to locate a relevant runbook from 9.8 to 1.6 minutes and a reduction in overall median time-to-resolution from 88.0 to 46.0 minutes across 187 tracked incidents. The article presents the graph schema, extraction and construction methodology, retrieval architecture, evaluation results, and discusses the organizational practices that determine whether such a graph remains a living, trustworthy source of institutional memory rather than a stale artifact.
Authors
- Venkata Praveen Annam
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-09-29
- DOI
- https://doi.org/10.5281/zenodo.23040082
- Primary Topic
- Software System Performance and Reliability
- Type
- article
- Field-Weighted Citation Impact
- 0.00