Simplification of Embedding Process in Retrieval Augmented Generation for Optimizing Question Answering Chatbot Model

In the current digital era, interaction through chatbots has become commonplace due to their ability to serve multiple users simultaneously. One of the widely used types of chatbots today is the Question Answering Chatbot (QAC). In this study, a QAC model is built using a Large Language Model (LLM) and the Retrieval Augmented Generation (RAG) framework. Compared to conventional generation models, RAG offers several advantages, such as strong scalability, easy data acquisition, and low training costs. The RAG process involves Indexing and Retrieval Generation stages. This research focuses on simplifying the embedding process in the indexing stage for efficient context management and response quality. This study utilizes GPT-3.5-Turbo as the LLM. The results obtained using GPT-3.5-Turbo indicate that the use of the Answer Only dataset and Prompt 2 provides the best response quality, with consecutive Faithfulness and Answer Relevancy metric values of 0.9167 and 0.9512, respectively. Subsequently, an experiment is conducted using the latest LLM, namely GPT-4o (omni), with Prompt 2 and the Question Only dataset to compare the two models. GPT-4o produces the best response quality with a Faithfulness metric value of $\mathbf{0 . 9 5 5 1}$. Meanwhile, the best Answer Relevancy score is achieved by GPT-3.5-Turbo with a value of 0.9512.