Improving Lessons Learned Retrieval in Project-based Organizations through Retrieval-Augmented Generation

Effective reuse of project knowledge is critical for improving performance and avoiding repeated errors in project-based organizations. Despite widespread adoption of lessons learned repositories, many organizations struggle to retrieve relevant insights due to the limitations of traditional keyword-based search. This study evaluates how retrieval-augmented generation (RAG) can enhance access to lessons learned in industrial knowledge repositories. A prototype system was developed using semantic embeddings, vector-based similarity search, local language models, and context-aware filtering, and tested on a real-world dataset from a leading industrial operator. Its performance was compared with the organization’s existing keyword-based search engine using quantitative metrics (Precision@10, Recall, F1 Score, MRR, nDCG@10) and qualitative feedback from domain experts across multiple project functions. Results show that the RAG approach substantially improved retrieval relevance and ranking, enabling faster access to critical lessons, while generative summarization reduced user effort by providing concise context-aware answers. The study also identified trade-offs between output quality and processing time and underscored the need for high-quality source documentation. Overall, the findings demonstrate the feasibility and value of locally deployed RAG solutions for improving knowledge reuse and organizational learning in project-intensive industries. The approach offers practical guidance for organizations seeking to improve knowledge reuse and project management efficiency through AI-enabled search and summarization.