Generative AI in the Taiwan medical laboratory scientist examination: Performance and educational utility

BACKGROUND: The rapid evolution of generative artificial intelligence (GAI), particularly large language models (LLMs), offers transformative potential for health professions education. However, their performance in specialized clinical domains, such as medical laboratory science as well as educational utility, remain insufficiently characterized. This study evaluated multiple LLMs on the Taiwanese Medical Laboratory Scientist Examination (TMLSE) and investigated the effectiveness of AI-generated assessments in undergraduate-year (UGY) medical laboratory training. METHODS: Four LLMs-GPT-3.5, GPT-4o, GPT-5.2, and Gemini 3 Pro-were evaluated using six TMLSE administrations from 2021 to 2023. Accuracy was assessed across 10 disciplines and image-based items, with incorrect responses categorized into informational, logical, or statistical errors. Additionally, 180 AI-generated items were validated by experts and administered to 30 UGY students. Student satisfaction and correlations between assessment performance and clinical internship outcomes were analyzed. RESULTS: All the models surpassed the 60% passing threshold, with clear generational improvement. GPT-5.2 (93.5%) and Gemini 3 Pro (92.1%) significantly outperformed GPT-4o (85.8%) and GPT-3.5 (67.1%). An error analysis indicated that informational deficiencies were the predominant failure mode, accounting for more than 95% of the incorrect responses across all the models. Although image-based items were more challenging than text-based questions were, GPT-5.2 demonstrated superior multimodal capability, achieving an image-based accuracy of 73.3%. In the educational evaluation, UGY students rated AI-generated items significantly higher than authentic TMLSE questions did in terms of overall satisfaction (p= 0.0089) and perceived utility for examination review (p= 0.0405). The performance on AI-generated assessments was positively correlated with clinical internship outcomes (Spearman's r= 0.3662, p= 0.0465), whereas no significant association was observed with traditional TMLSE scores. CONCLUSION: Modern LLMs demonstrate high performance in specialized medical laboratory knowledge, although image interpretation remains a limitation. Integrating AI-generated assessments into UGY curricula represents an effective and scalable approach to competency-based education, with demonstrated alignment to real-world clinical performance.

Authors

Institutions

Publication Details

Journal
Journal of the Chinese Medical Association
Published
2026-11-12
DOI
https://doi.org/10.1097/jcma.0000000000001445
Primary Topic
Artificial Intelligence in Healthcare and Education
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Generative AI in the Taiwan medical laboratory scientist examination: Performance and educational utility

Hao-Ru Lee, Fangyu Wang, Hsiang‐Ling Ho, Chieh-Hsi Wu et al.
Journal of the Chinese Medical Association
Artificial Intelligence in Healthcare and Education
article

Generative AI in the Taiwan medical laboratory scientist examination: Performance and educational utility

Hao-Ru Lee, Fangyu Wang, Hsiang‐Ling Ho, Chieh-Hsi Wu, Yen-Li Wang
article en

Abstract

BACKGROUND: The rapid evolution of generative artificial intelligence (GAI), particularly large language models (LLMs), offers transformative potential for health professions education. However, their performance in specialized clinical domains, such as medical laboratory science as well as educational utility, remain insufficiently characterized. This study evaluated multiple LLMs on the Taiwanese Medical Laboratory Scientist Examination (TMLSE) and investigated the effectiveness of AI-generated assessments in undergraduate-year (UGY) medical laboratory training. METHODS: Four LLMs-GPT-3.5, GPT-4o, GPT-5.2, and Gemini 3 Pro-were evaluated using six TMLSE administrations from 2021 to 2023. Accuracy was assessed across 10 disciplines and image-based items, with incorrect responses categorized into informational, logical, or statistical errors. Additionally, 180 AI-generated items were validated by experts and administered to 30 UGY students. Student satisfaction and correlations between assessment performance and clinical internship outcomes were analyzed. RESULTS: All the models surpassed the 60% passing threshold, with clear generational improvement. GPT-5.2 (93.5%) and Gemini 3 Pro (92.1%) significantly outperformed GPT-4o (85.8%) and GPT-3.5 (67.1%). An error analysis indicated that informational deficiencies were the predominant failure mode, accounting for more than 95% of the incorrect responses across all the models. Although image-based items were more challenging than text-based questions were, GPT-5.2 demonstrated superior multimodal capability, achieving an image-based accuracy of 73.3%. In the educational evaluation, UGY students rated AI-generated items significantly higher than authentic TMLSE questions did in terms of overall satisfaction (p= 0.0089) and perceived utility for examination review (p= 0.0405). The performance on AI-generated assessments was positively correlated with clinical internship outcomes (Spearman's r= 0.3662, p= 0.0465), whereas no significant association was observed with traditional TMLSE scores. CONCLUSION: Modern LLMs demonstrate high performance in specialized medical laboratory knowledge, although image interpretation remains a limitation. Integrating AI-generated assessments into UGY curricula represents an effective and scalable approach to competency-based education, with demonstrated alignment to real-world clinical performance.

Journal of the Chinese Medical Association
National Yang Ming Chiao Tung University (TW), Taipei Veterans General Hospital (TW)
Quality education, Good health and well-being
Artificial Intelligence in Healthcare and Education
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon