CPEQA: A Large Language Model Based Knowledge Base Retrieval System for Chinese Confidentiality Knowledge Question Answering

Large language models (LLMs) have exhibited remarkable performance on various natural language processing (NLP) tasks, particularly in the construction of the intelligent question-answering system. These systems, especially in specialized fields, usually rely on NLP through the retrieval of corpus and answering databases to efficiently provide accurate and concise answers. This paper focuses on the national confidentiality publicity and education field, aiming to address the dilemma of inaccurate knowledge retrieval in this field. Therefore, we design an intelligent confidentiality question-answering system CPEQA by comprehensively utilizing the LLMs platform and information retrieval technique. CPEQA is capable of providing professional answers to questions about Chinese confidentiality publicity and education raised by users. Additionally, we also integrate the conventional database retrieval technique and LLMs into the database query construction, enabling CPEQA to perform real-time queries and data analysis for both single-table and multi-table querying tasks. Through extensive experiments with generated query sentences, we show both methodological comparisons and empirical evaluations of CPEQA’s performance. Experimental results indicate that CPEQA has achieved competitive results on answering precision, recall rate and other metrics. Finally, we explore the challenges of the CPEQA system associated with these techniques and outline potential avenues for future research in this emerging field.