A Comparative Study of Vietnamese Embedding Models for LLM-Based Automated Question Answering on DUT Regulations

Question Answering (QA) is a specialized subfield within Information Retrieval (IR) that focuses on providing precise answers to user queries. Recent advances in Large Language Models (LLMs) have greatly improved the speed and accuracy of question-answering (QA) systems, leading to significantly enhanced capabilities and performance. This study introduces a multi-PDF querying system designed to support automated question answering for Vietnamese students, leveraging the capabilities of LLMs. To enhance the retrieval accuracy from natural language queries, we integrate four Vietnamese-language embedding models. These models are evaluated using a dataset comprising questions related to the academic regulations of the University of Da Nang - University of Science and Technology (DUT), reflecting real-life scenarios students commonly encounter. Experimental results reveal that the top-performing Vietnamese embedding model achieves a Mean Average Precision (mAP) of 0.840 and an F1-score of 0.959, demonstrating its high effectiveness in information retrieval. This study makes significant contributions to the integration of Vietnamese embedding models into automated question-answering systems, aiming to effectively support students in the university environment.