Enhancing Data Traceability: A Knowledge Graph Approach with Retrieval-Augmented Generation

This study explores the challenges of data traceability within Company ABC’s Master Data Management (MDM) system, where handling data-related requests has become increasingly complex due to inefficient manual processes. Despite MDM’s implementation, business growth has led to the accumulation of documents and difficulties in tracking request lineage, causing bottlenecks in daily operations. The study proposes a solution that integrates knowledge graphs with Retrieval-Augmented Generation (RAG), inspired by LinkedIn’s question-answering system, to improve data management and traceability. The knowledge graph is constructed by manually parsing spreadsheet data related to requests, allowing for more efficient data integration and retrieval. The study compares three approaches—graph-based, vector-based, and hybrid—in terms of answer correctness and context retrieval. The graph-based approach yielded the best performance, improving answer correctness by 21.09% over the vector-based method and by 19.34% over the hybrid method. While the hybrid method achieved the highest context precision at 83.49%, it struggled with context recall, retrieving relevant context only 69.09% of the time. The study also found that the vector-based method’s retrieval of irrelevant context caused the language model to produce misleading answers, even when accompanied by more relevant graph-based context. In conclusion, integrating RAG with knowledge graphs significantly enhances answer accuracy and context retrieval precision. Future research should focus on refining context-weighting techniques between the vector and graph-based methods and on addressing the issue of irrelevant context to further improve the overall effectiveness of the model in data traceability and information retrieval tasks.