Tillämpning av Modulär RAG-Arkitektur i en Chattassistent för Navigering av Verkstadsplanering : En Implementering och Utvärdering av Två RAG-Flödesmönster

This thesis presents the implementation of a chatbot assistant utilizing the modular Retrieval-Augmented Generation (RAG) architecture, specifically tailored for use in a car repair shop setting. The system leverages the generative capabilities of state-of-the-art large language models (LLMs) while extending non-parametric memory through semantic retrieval to improve response quality. Two architectural variants of the modular RAG pattern are designed and evaluated: a branching pattern and an iterative pattern. The practical implementation is followed by a theoretical investigation that compares the two architectural patterns across multiple metrics, including retrieval quality, generation quality, response speed, and cost analysis. Evaluation is performed using a test set of queries with the Ragas evaluation framework. Six metrics are evaluated to reflect a wide range of qualities related to retrieval and generation. Analysis of the results reveals comparable performance in answer correctness, but the branching pattern gives improvements of +8% in answer relevance and +5% in semantic similarity. More significant differences are found in the retrieval metrics, with the branching pattern achieving a +18% increase in context recall. Furthermore, the branching pattern responds 13% faster and has a 50% lower average cost compared to the iterative pattern. The results indicate that a modular RAG architecture is effective in implementing a chatbot assistant for question answering (QA) using internal data. The theoretical investigation reveals that the branching pattern is better suited for applications utilizing smaller data sources, as the benefits of the iterative pattern found in previous research are not replicated in this study.