Innovative Approach of Generative AI for Automating Technical Bid Evaluations in Oil Companies

Summary The process of outlining a scope of work and evaluating technical bids in the oil and gas industry is commonly burdensome, labor-intensive, and susceptible to human bias. This paper introduces an AI-assistant chatbot based on the power of open-source large language models (LLMs), natural language processing (NLP), and data analytics, to aid in automating the entire workflow of technical tendering processes, facilitating an improved decision support system (DSS), and mitigating potential subjectivity. The workflow starts with loading documents in the format of scanned PDF files. Firstly, data was extracted using optical character recognition (OCR), and text mining techniques. Secondly, Langchain was implemented to optimally split the document into chunks with adequate overlapping. Thirdly, embeddings were created using sentence transformers, and a vector store was constructed. After that, LlaMa model, was optimized and employed to query the vector store efficiently. Then, retrieval augmented generative (RAG) query engines were used to retrieve the matching documents and generate the final answer. Finally, a chat memory buffer was incorporated to maintain context and initialize a chatbot. Implementing LLM in the tendering process has proven promising for reviewing and ranking the technical data of the submitted bids based on pre-defined criteria. However, it was shown that extracting tabulated data effectively from scanned documents is challenging. Consequently, it was essential to pre-process documents and extract tables into structured databases before querying them. In addition, model augmentation was employed to avoid hallucination and enhance the model reasoning and capability of identifying pass and fail criteria, nonetheless, human feedback is required to assess satisfying performance. Recall-Oriented Understudy for Gisting Evaluation (ROUGE) was found valuable to expedite the comparison between the submitted data in technical tables and the detailed data sheets. Furthermore, LLM was found effective in accelerating the process of market survey comparison, by retrieving and comparing prices and conditions of different documents. Moreover, supervised fine-tuning (SFT) was implemented to improve the model's capacity to compare clauses of the scope of work along with terms and conditions across different contracts, in order to discern major inherent differences and reduce potential bias in the future. This enhancement was configured through quantized low-rank adaptation (QLoRA) and parameter-efficient fine-tuning (PEFT). This paper presented a novel approach, which integrated LLMs into procurement information development systems, automating multiple tasks in the tendering process in the oil and gas upstream industry. This is the first time that such an application has been applied to the oil industry, which shall provide a foundation for future research.