Building an Intelligent Research Assistant – RAG Isn't All You Need

Abstract Based on experimentation, this paper outlines the efficiency and accuracy improvements that can be achieved using a cutting-edge generative AI chat enabled research assistant to facilitate well planning exercises. The technology innovations required to achieve those gains are also outlined. With the goal of measuring these improvements, during the summers of 2024 and 2025, the research team tested an experimental, domain aware agentic research assistant which could answer a question, cite the sources and show the researcher the original material it had used to determine the answer. Seventy-five (75) volunteers from thirty (30) countries participated in a simple well planning exercise answering a series of questions requiring analysis and interpolation of information contained in Equinor's Volve data set which consists of approximately 30,000 documents of mostly unstructured data (PDFs, etc.). To assist in answering the questions, the system had access to a good portion of those documents including well design documents, well logs, geological and stratigraphical data and operational reports (e.g. daily drilling reports) as well as SPE's PetroWiki. The system was benchmarked against a more standard but still state of the art RAG (Retrieval Augmented Generation) solution. By all measures, the experimental system outperformed the state-of-the-art RAG system, improving the accuracy of the answers by 152% and reducing the time required to answer by 86%. This paper explores the efficiency gained when a researcher, especially one with less industry experience, has access to a capable virtual research assistant which, in turn, has access to a broad set of documents covering the domain. The paper describes the limits of a state-of-the-art RAG system, the impact of building domain awareness into a novel system and the technology areas that make a difference in that system's usefulness when presented with the volumes of material common in the industry. While the technology continued to be improved in general, the next round of research focused on compiling answers across a large set of documents and finding information within plots and graphs. Similar testing, this time with testers who did not have oil and gas experience, showed that subjects achieved 86% correct answers with the improved solution vs 27% with an updated baseline RAG system. Finally, we performed ablation testing to understand the efficacy of various search approaches in achieving these results for various types of questions.