An Empirical Multidimensional Evaluation of Response Quality in Medical Automatic Question-Answering Systems

This study addresses the challenge of evaluating response quality in medical automatic question-answering (QA)systems. Based on an established multidimensional index framework, we conduct an empirical analysis of ZuoYi-GPT using real-world medical QA data. Text mining and natural language processing techniques are employed to quantitatively extract indicator values, and an improved CRITIC-based objective weighting method is applied for indicator weighting. The results reveal that the accuracy dimension carries the highest weight (87.58%), and the overall response quality is relatively high, with high- and medium-quality responses accounting for 84.21% of the total. However, shortcomings are observed in aspects such as medical terminology density, syntactic diversity, and structural completeness. The findings provide empirical support for the targeted optimization of AI-based medical applications and offer methodological insight for information quality assessment in other vertical domains under the development of large-scale language models.