VQABG: Vietnamese question/answers benchmark generator for field-specific chatbot ground-truth dataset using EMINI (Exact Match wIth Numeric Information) indicator

Currently, the application of generative Artificial Intelligence for developing specialized chatbots in Vietnamese is an inevitable trend. However, one of the most challenging aspects of assessing the quality of Vietnamese chatbot products is creating a specialized benchmark in a question-and-answer format. Typically, this benchmark is manually crafted by industry experts, which can be extremely costly. In contrast, for English, we can use bag-of-words model toolkits and grammatical structure architectures to generate appropriate questions automatically based on pre-existing answers from the original data. However, there is almost no complete model available for this task in Vietnamese. Regarding quality assessment, this is usually performed manually by experts using Human Evaluation (HE) indicators, which is also costly. Therefore, the aim of this study is to propose an algorithmic architecture specifically designed for the Vietnamese language. This architecture will automatically generate a set of question-and-answer queries to create a benchmark, as well as facilitate the development of a mechanism for automatic, straightforward, cost-effective, and accurate quality assessment for Vietnamese chatbots. We refer to this system as the Vietnamese Question/Answers Benchmark Generator (VQABG) and propose an innovative evaluation indicator called the Exact Match with Numeric Information (EMINI).