Comparative performance of DeepSeek-R1 and DeepSeek-V3 on China’s National Medical Licensing Examination questions (2004–2022)

Background DeepSeek, a cutting-edge artificial intelligence (AI) platform developed in China, has attracted growing interest for potential applications in clinical medicine and medical education. DeepSeek-V3 (MoE architecture) and DeepSeek-R1 (reinforcement learning-optimized for reasoning) are its flagship models, but the performance difference between DeepSeek‑R1 and DeepSeek‑V3 remains inconclusive. The objective of this study was to compare the performance of DeepSeek-R1 and DeepSeek-V3 on the Chinese National Medical Licensing Examination (NMLE). Methods This study employed DeepSeek-R1-0528 (released on May 28, 2025) and DeepSeek-V3-0324 (released on March 24, 2025) to perform the NMLE testing, and the subject area for the examination was Clinical Medicine. We collected the questions from NMLE 2004 through NMLE 2022, comprising 600 multiple-choice questions (MCQs) per annual examination, for a total of 11,400 MCQs. The NMLE tests were carried out using the free web-based version of DeepSeek, with DeepSeek-R1-0528 and DeepSeek-V3-0324 evaluated on separate computer systems. Model performance was assessed based on accuracy rates. Results The annual average accuracy rates for DeepSeek-R1-0528 and DeepSeek-V3-0324 were 91.2% (95% CI: 89.6% – 92.8%) and 91.1% (95% CI: 89.6% – 92.6%), respectively, with no significant difference observed between them ( P = 0.591, Cohen’s d = 0.086). The regression analysis showed that DeepSeek-R1-0528 showed no significant changes in accuracy rates from NMLE 2004 to NMLE 2013, while DeepSeek-V3-0324 showed a significant declining trend. Moreover, from NMLE 2014 to NMLE 2022, both models demonstrated significant declining trends in the accuracy rates. DeepSeek-R1-0528 and DeepSeek-V3-0324 answered 999 and 1,014 MCQs incorrectly, respectively, with 715 MCQs answered incorrectly by both models. Conclusions DeepSeek-R1-0528 and DeepSeek-V3-0324 demonstrated near-identical and excellent performance on the Chinese NMLE.

Authors

Institutions

Publication Details

Journal
PLoS ONE
Published
2026-11-19
DOI
https://doi.org/10.1371/journal.pone.0361855
Primary Topic
Artificial Intelligence in Healthcare and Education
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Comparative performance of DeepSeek-R1 and DeepSeek-V3 on China’s National Medical Licensing Examination questions (2004–2022)

Zhuojun Li, Qian-Qian Liao, Ke‐Wei Zhu, Zhou-Ni Xie et al.
PLoS ONE
Artificial Intelligence in Healthcare and Education
article

Comparative performance of DeepSeek-R1 and DeepSeek-V3 on China’s National Medical Licensing Examination questions (2004–2022)

Zhuojun Li, Qian-Qian Liao, Ke‐Wei Zhu, Zhou-Ni Xie, Yan-Feng Tang, Yu-Jiao Lin, Run Ge, Yang-Chun Li
article en

Abstract

Background DeepSeek, a cutting-edge artificial intelligence (AI) platform developed in China, has attracted growing interest for potential applications in clinical medicine and medical education. DeepSeek-V3 (MoE architecture) and DeepSeek-R1 (reinforcement learning-optimized for reasoning) are its flagship models, but the performance difference between DeepSeek‑R1 and DeepSeek‑V3 remains inconclusive. The objective of this study was to compare the performance of DeepSeek-R1 and DeepSeek-V3 on the Chinese National Medical Licensing Examination (NMLE). Methods This study employed DeepSeek-R1-0528 (released on May 28, 2025) and DeepSeek-V3-0324 (released on March 24, 2025) to perform the NMLE testing, and the subject area for the examination was Clinical Medicine. We collected the questions from NMLE 2004 through NMLE 2022, comprising 600 multiple-choice questions (MCQs) per annual examination, for a total of 11,400 MCQs. The NMLE tests were carried out using the free web-based version of DeepSeek, with DeepSeek-R1-0528 and DeepSeek-V3-0324 evaluated on separate computer systems. Model performance was assessed based on accuracy rates. Results The annual average accuracy rates for DeepSeek-R1-0528 and DeepSeek-V3-0324 were 91.2% (95% CI: 89.6% – 92.8%) and 91.1% (95% CI: 89.6% – 92.6%), respectively, with no significant difference observed between them ( P = 0.591, Cohen’s d = 0.086). The regression analysis showed that DeepSeek-R1-0528 showed no significant changes in accuracy rates from NMLE 2004 to NMLE 2013, while DeepSeek-V3-0324 showed a significant declining trend. Moreover, from NMLE 2014 to NMLE 2022, both models demonstrated significant declining trends in the accuracy rates. DeepSeek-R1-0528 and DeepSeek-V3-0324 answered 999 and 1,014 MCQs incorrectly, respectively, with 715 MCQs answered incorrectly by both models. Conclusions DeepSeek-R1-0528 and DeepSeek-V3-0324 demonstrated near-identical and excellent performance on the Chinese NMLE.

PLoS ONEVol. 21(11)
Guilin Medical University (CN), Guangdong Baiyun University (CN)
Artificial Intelligence in Healthcare and Education
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon