GVEC: Generative Vietnamese Economy Chatbots using Vietnamese Numeric Economy Information Question/ Answer Dataset (VNEIQAD) benchmark

Currently, the problem of automatic question answering (QA) in chatbot services has become an essential need, supporting various aspects of daily life. In this study, we aim to study this problem by applying QA techniques to the context of the Vietnamese economy. The chatbot system, named Generative Vietnamese Economy Chatbot (GVEC), is based on the Vietnam Economy Information Database (VElD) retrieved from the VnEconomy systems. We propose comparing the application of Retrieval-Augmented Generation (RAG) to various large language models (LLMs) for a question-answering system. This will be tested using our proposed benchmark, the Vietnamese Numeric Economy Information Question/Answer Dataset (VNEIQAD), which is generated by the Question/Answer Generation with Numerical Information (QAGwNI) algorithm and has advantages in the evaluation task, especially in the economic context instead of the traditional method. For evaluation, we use RAGAS (a collection of RAG evaluation metrics), as well as our proposed NIEM (Numeric Information Exact Match), in conjunction with sample responses to analyze the evaluation of content generated by chatbots.