Research on Multi-Agent Question Answering Based on Large Language Models

The continuous development of large language models has led to tremendous performance improvement and has made them increasingly more important. More and more researchers are starting to explore the use of these large language models in developing artificial intelligence agents. The goal of this research is to develop a question-answering system based on a multi-agent setup with the open-source large language model Qwen2.5. In this multi-agent setup, the two agents include a student and a teacher where the student solves issues with guidance from the teacher. We evaluate accuracy difference between a single-agent and a multi-agent setup with the Chinese evaluation data, C-Eval. This is a departure from other approaches like fine-tuning large models or utilizing methods like chain of thought (CoT) and n-shot learning that are tailor-made to improve model performance. Rather, it is a more general approach aimed at further testing the abilities of large language models and thereby improve their logical reasoning capabilities. The experimental results demonstrate a significant improvement in accuracy in multi-agent systems compared to single-agent setup. In particular, improvement in accuracy over various subjects is around 2.5%. In addition, it has been found that improvement is directly proportional to model parameter size; for example, a model with a parameter size of 14 billion has a 22.18% improvement in accuracy for hard cases and reflects a significant improvement in performance.