AcaPigeon: A Large Language Model-Based Academic Information Consultation Assistant

Academic Information Services are a crucial component of higher education, integral throughout the student’s learning journey. However, issues still persist regarding the convenience and intelligence of student information retrieval. To address these problems, we propose an academic information consultation assistant centered around a large language model, offering more convenient and intelligent services through natural language interaction. Specifically, we have designed a three-stage framework. Firstly, to identify the information sources to be retrieved, we utilize Few-Shot and other prompt engineering techniques with the large language model to analyze and annotate the queries. Secondly, based on the identified information sources, to enhance retrieval hit rates and thereby improve answer accuracy, different sources are equipped with customized retrievers suited to their unique characteristics. Lastly, we establish strict response standards, generating accurate answers through clear prompts and adhering to the principle of minimizing misleading information. Experiments demonstrate that our approach significantly outperforms baseline methods, with substantial improvements in hit rate and accuracy.