Evaluating Large‐Scale and Lightweight Large Language Models for Traditional Chinese Medicine Exam Questions: A Comparative Study
This study demonstrates that although large-scale LLMs exhibit strong knowledge recall in TCM, their suboptimal performance on multiple-choice questions and substantial computational costs may limit their practical applicability in clinical settings. The robust performance of Qwen3-1.7B indicates that effective model optimization and domain-specific training may offer greater advantages than simply increasing model size. While the current evaluation is based on examination-style tasks and does not involve real-world clinical decision-making, our findings provide insights to support the deployment of optimized models in resource-constrained healthcare environments.
Authors
- Shaohan Huang (ORCID: https://orcid.org/0000-0003-4324-6337)
- Lu Yin (ORCID: https://orcid.org/0000-0002-5075-5263)
- Zhongzhi Luan
- Xincan Liu
- Dongran Han
- Lei Quan
- Bin Li
- Jiaxing Qi
- Yizhen Li
Institutions
- Beijing University of Chinese Medicine (CN)
- First Affiliated Hospital of Henan University (CN)
- Beihang University (CN)
Publication Details
- Journal
- Journal of Evidence-Based Medicine
- Published
- 2026-02-11
- DOI
- https://doi.org/10.1111/jebm.70118
- Primary Topic
- Traditional Chinese Medicine Studies
- Type
- article
- Field-Weighted Citation Impact
- 0.00