Empowering Business Decisions and Knowledge Management Through Advanced RAG-Driven QA Systems

In recent decades, the rapid evolution of Artificial Intelligence (AI) has revolutionized business processes, enabling significant improvements in knowledge management, analytics, and decision-making. This paper explores the application of Advanced Retrieval-Augmented Generation (RAG) techniques in corporate databases to enhance data exploration and decision support in business environments through the development of a Question Answering (QA) system. The QA system is designed to assist with technical problem-solving and business information management in a case study conducted within a consulting firm. To process the diverse resources, we employed OpenAl's Whisper for audio transcription, while we used Python's Un-structured library alongside Pytesseract and OpenCV to perform document parsing and Optical Character Recognition (OCR). We evaluated the QA system powered by OpenAl's GPT-4o-mini using a range of Query Augmentation techniques including Hyde, Query Rewrite, Rephrasal, and Step Back Prompting, as well as Reranking methods such as mT5, BM25, BGE m3, and RankGPT. To evaluate the system, we employed RAGAS framework to create a meaningful test set and to assess the performance, using Answer Relevancy and Faithfulness. The combination of Step Back Prompting and RankGPT achieved an improvement of 19.9% in Answer Relevancy compared to Naive RAG, marking a significant advancement over traditional Large Language Models (LLMs) without RAG integration. This study highlights the potential of RAG-enhanced QA solutions for advancing knowledge management and decision support in business contexts.