Large Language Models and Applied Surgical Anatomy: Evaluation of Anatomical Reasoning in Expert‐Derived Plastic Surgery Scenarios

ABSTRACT Large language models (LLMs) are increasingly being integrated into medical education, research, and clinical decision‐support applications; however, their ability to apply anatomical knowledge in clinically relevant surgical scenarios remains insufficiently studied. This study evaluated and compared the performance of six contemporary LLMs (ChatGPT, Gemini, Claude, DeepSeek, Copilot, and Manus) using 25 expert‐derived applied surgical anatomy scenarios relevant to plastic surgery. Scenarios were developed by an expert panel consisting of one associate professor of anatomy and two plastic surgeons. Responses were generated using a standardized prompt and assessed with a structured scoring rubric evaluating anatomical accuracy, clinical reasoning, completeness, and safety, with a maximum score of 10 points per question. Critical errors were additionally assessed. Significant differences in overall performance were observed among models (Friedman χ 2 = 43.51, p < 0.001). ChatGPT achieved the maximum aggregate score (250/250), followed by Gemini (248/250), DeepSeek (241/250), Copilot (240/250), Claude (235/250), and Manus (219/250). Critical errors were infrequent and were confined to a reconstructive vascular physiology scenario: DeepSeek, Claude, and Manus each demonstrated one observed critical error, whereas ChatGPT, Gemini, and Copilot demonstrated none. Although all evaluated models demonstrated excellent performance, clinically significant anatomical misconceptions persisted despite high aggregate scores. These findings demonstrate strong performance by the tested LLMs on this specific scenario‐based benchmark, while also identifying clinically meaningful anatomical errors in a subset of responses. Future evaluations should focus not only on overall accuracy but also on clinically meaningful error patterns that may influence surgical decision‐making. Importantly, substantial ceiling effects limited discrimination among high‐performing models, and the findings should therefore be interpreted as performance on this specific benchmark rather than as evidence of a general hierarchy of anatomical reasoning capability.

Authors

Institutions

Publication Details

Journal
Clinical Anatomy
Published
2026-10-26
DOI
https://doi.org/10.1002/ca.70243
Primary Topic
Artificial Intelligence in Healthcare and Education
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Large Language Models and Applied Surgical Anatomy: Evaluation of Anatomical Reasoning in Expert‐Derived Plastic Surgery Scenarios

Süleyman Çeçen, Yadigar Kastamonı, Menekşe Kastamoni
1 links
Clinical Anatomy
Artificial Intelligence in Healthcare and Education
article

Large Language Models and Applied Surgical Anatomy: Evaluation of Anatomical Reasoning in Expert‐Derived Plastic Surgery Scenarios

Süleyman Çeçen, Yadigar Kastamonı, Menekşe Kastamoni
article en

Abstract

ABSTRACT Large language models (LLMs) are increasingly being integrated into medical education, research, and clinical decision‐support applications; however, their ability to apply anatomical knowledge in clinically relevant surgical scenarios remains insufficiently studied. This study evaluated and compared the performance of six contemporary LLMs (ChatGPT, Gemini, Claude, DeepSeek, Copilot, and Manus) using 25 expert‐derived applied surgical anatomy scenarios relevant to plastic surgery. Scenarios were developed by an expert panel consisting of one associate professor of anatomy and two plastic surgeons. Responses were generated using a standardized prompt and assessed with a structured scoring rubric evaluating anatomical accuracy, clinical reasoning, completeness, and safety, with a maximum score of 10 points per question. Critical errors were additionally assessed. Significant differences in overall performance were observed among models (Friedman χ 2 = 43.51, p < 0.001). ChatGPT achieved the maximum aggregate score (250/250), followed by Gemini (248/250), DeepSeek (241/250), Copilot (240/250), Claude (235/250), and Manus (219/250). Critical errors were infrequent and were confined to a reconstructive vascular physiology scenario: DeepSeek, Claude, and Manus each demonstrated one observed critical error, whereas ChatGPT, Gemini, and Copilot demonstrated none. Although all evaluated models demonstrated excellent performance, clinically significant anatomical misconceptions persisted despite high aggregate scores. These findings demonstrate strong performance by the tested LLMs on this specific scenario‐based benchmark, while also identifying clinically meaningful anatomical errors in a subset of responses. Future evaluations should focus not only on overall accuracy but also on clinically meaningful error patterns that may influence surgical decision‐making. Importantly, substantial ceiling effects limited discrimination among high‐performing models, and the findings should therefore be interpreted as performance on this specific benchmark rather than as evidence of a general hierarchy of anatomical reasoning capability.

Clinical Anatomy
Bursa Uludağ Üni̇versi̇tesi̇ (TR), Süleyman Demirel Üniversitesi (TR)
Openalex Percentile: Top 19%
Artificial Intelligence in Healthcare and Education
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon