Benchmarking Large Language Models on the Turkish Dermatology Board Exam: A Comparative Multilingual Analysis

Artificial intelligence (AI) technologies are used as an effective tool in medical education to support the acquisition of theoretical knowledge and improve the clinical skills of medical students and residents. 1The increasing use of AI applications in medical education has potential, especially in disciplines based on visual diagnoses, such as dermatology; this trend necessitates re-evaluating conventional assessment tools, such as specialty competency exams, with AI models. 2 Studies on the performance of AI models in medical education exams reveal the potential, limitations, and room for improvement of this technology. 3Recent benchmark studies show that state-of-the-art large language models (LLMs) (e.g., GPT-4, Gemini Advanced, Claude) can exceed the 60% pass mark on united states medical licensing examination step 1-style items and perform at or near the resident level in ophthalmology and orthopedic vignette sets. [3][4][5] Despite the progress made, two critical knowledge gaps remain.First, almost all validation studies have been conducted in English, while more than half of the world's medical