Evaluating Large‐Scale and Lightweight Large Language Models for Traditional Chinese Medicine Exam Questions: A Comparative Study

This study demonstrates that although large-scale LLMs exhibit strong knowledge recall in TCM, their suboptimal performance on multiple-choice questions and substantial computational costs may limit their practical applicability in clinical settings. The robust performance of Qwen3-1.7B indicates that effective model optimization and domain-specific training may offer greater advantages than simply increasing model size. While the current evaluation is based on examination-style tasks and does not involve real-world clinical decision-making, our findings provide insights to support the deployment of optimized models in resource-constrained healthcare environments.

Authors

Institutions

Publication Details

Journal
Journal of Evidence-Based Medicine
Published
2026-02-11
DOI
https://doi.org/10.1111/jebm.70118
Primary Topic
Traditional Chinese Medicine Studies
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Evaluating Large‐Scale and Lightweight Large Language Models for Traditional Chinese Medicine Exam Questions: A Comparative Study

Shaohan Huang, Lu Yin, Zhongzhi Luan, Xincan Liu et al.
Journal of Evidence-Based Medicine
Traditional Chinese Medicine Studies
article

Evaluating Large‐Scale and Lightweight Large Language Models for Traditional Chinese Medicine Exam Questions: A Comparative Study

Shaohan Huang, Lu Yin, Zhongzhi Luan, Xincan Liu, Dongran Han, Lei Quan, Bin Li, Jiaxing Qi, Yizhen Li
article en

Abstract

This study demonstrates that although large-scale LLMs exhibit strong knowledge recall in TCM, their suboptimal performance on multiple-choice questions and substantial computational costs may limit their practical applicability in clinical settings. The robust performance of Qwen3-1.7B indicates that effective model optimization and domain-specific training may offer greater advantages than simply increasing model size. While the current evaluation is based on examination-style tasks and does not involve real-world clinical decision-making, our findings provide insights to support the deployment of optimized models in resource-constrained healthcare environments.

Journal of Evidence-Based Medicine
Beijing University of Chinese Medicine (CN), First Affiliated Hospital of Henan University (CN), Beihang University (CN)
Openalex Percentile: Top 56%
Traditional Chinese Medicine Studies
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.