Agent- och RAG-baserat verktyg för identifiering och sammanfattning av fel i industriella byggpipelines

Root cause analysis (RCA) is the process of identifying and solving underlying issues that cause failures and is an important part of maintaining embedded systems. The analysis can often be time-consuming, resource-intensive, and tedious in complex systems, especially when performed manually. Automating this process could save time and reduce effort. While there currently exist tools for RCA in different systems and areas, Large Language Models (LLMs) have more recently been incorporated into these tools to further automate and facilitate the process. This thesis investigates how a tool based on LLMs, retrieval-augmented generation (RAG), and prompt techniques can automate and assist with identifying and summarizing the root cause based on log files and source code. Furthermore, it explores how accurately the tool can identify and summarize the root cause. In this thesis, a tool containing two summarizers was developed. Both summarizers used prompt techniques and an LLM but differed in their architecture. One summarizer was based on RAG, while the other was based on an agentic approach. The agentic summarizer iteratively investigated logs and code using tools. The summarizers were assessed quantitatively, where users evaluated and rated the generated summaries through a user interface to determine the accuracies of the summarizers. The tool was also evaluated through qualitative data, where feedback on usefulness, interpretability, and speed was collected through a user survey. The results showed that both summarizers achieved an accuracy of over 80%. The agentic summarizer slightly exceeded the RAG-based summarizer. However, the difference was not statistically significant. Qualitative analysis indicated that the agentic summarizer was perceived as more insightful and aligned better with how RCA is performed in real-life scenarios, even though it was slower and occasionally made assumptions. The tool was seen as valuable, especially for reducing time spent on log analysis.