Frontier Large Language Models as Adjunctive Perioperative Decision-Support Tools in Elective Gastrointestinal Surgery: A Prospective Longitudinal Comparison With Consultant-Led Multidisciplinary Care

BackgroundFrontier large language models (LLMs) are emerging as perioperative clinical decision-support tools, but prospective comparisons with consultant-led multidisciplinary teams (MDTs) remain limited. We evaluated the inter-model agreement, concordance with consultant-led multidisciplinary care, safety, and assessment-level inter-model consensus of four frontier LLMs in elective gastrointestinal surgery within an Enhanced Recovery After Surgery (ERAS) pathway.MethodsIn this prospective longitudinal observational study, 250 adults contributed 2398 longitudinal clinical assessments and 9592 individual LLM recommendations from preoperative evaluation through discharge. Each assessment was independently evaluated by ChatGPT, Claude, Gemini, and DeepSeek using identical locked prompts in separate conversations without memory retention or response regeneration. Recommendations were compared with the consultant-led MDT clinical reference standard. Primary outcome was MDT agreement; secondary outcomes included Cohen's κ, domain-specific agreement, assessment-level inter-model consensus, hallucinations, unsafe recommendations, and longitudinal performance using mixed-effects regression with generalized estimating equation and bootstrap sensitivity analyses.ResultsA total of 9592 independent LLM evaluations were analyzed. Overall agreement was highest for Gemini (93.2%; κ = 0.87), followed by Claude (92.9%; κ = 0.86), ChatGPT (91.8%; κ = 0.84), and DeepSeek (88.8%; κ = 0.79). Complete inter-model consensus occurred in 1834 assessments (76.5%), majority consensus in 352 (14.7%), split consensus in 154 (6.4%), and complete disagreement in 58 (2.4%). Of 799 recommendations discordant with the MDT reference standard, 605 met the prespecified criteria for formal adjudication. An additional 95 recommendations were randomly selected from those not meeting the formal criteria, yielding 700 recommendations for blinded expert adjudication. Confirmed safety events were uncommon (2.6-6.0% of evaluations).ConclusionsFrontier LLMs demonstrated high concordance with consultant-led MDT recommendations under standardized prompting conditions. Greater assessment-level inter-model consensus was associated with higher MDT concordance. These findings support further prospective evaluation of LLMs as supervised adjunctive perioperative decision-support tools but do not establish clinical correctness, outcome benefit, or suitability for autonomous decision-making.

Authors

Institutions

Publication Details

Journal
The American Surgeon
Published
2026-11-17
DOI
https://doi.org/10.1177/00031348261507472
Primary Topic
Artificial Intelligence in Healthcare and Education
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Frontier Large Language Models as Adjunctive Perioperative Decision-Support Tools in Elective Gastrointestinal Surgery: A Prospective Longitudinal Comparison With Consultant-Led Multidisciplinary Care

Rudraksh Kesharwani, Arundati Ramdas Raikar, Anushka Singh
4 links
The American Surgeon
Artificial Intelligence in Healthcare and Education
article

Frontier Large Language Models as Adjunctive Perioperative Decision-Support Tools in Elective Gastrointestinal Surgery: A Prospective Longitudinal Comparison With Consultant-Led Multidisciplinary Care

Rudraksh Kesharwani, Arundati Ramdas Raikar, Anushka Singh
article en

Abstract

BackgroundFrontier large language models (LLMs) are emerging as perioperative clinical decision-support tools, but prospective comparisons with consultant-led multidisciplinary teams (MDTs) remain limited. We evaluated the inter-model agreement, concordance with consultant-led multidisciplinary care, safety, and assessment-level inter-model consensus of four frontier LLMs in elective gastrointestinal surgery within an Enhanced Recovery After Surgery (ERAS) pathway.MethodsIn this prospective longitudinal observational study, 250 adults contributed 2398 longitudinal clinical assessments and 9592 individual LLM recommendations from preoperative evaluation through discharge. Each assessment was independently evaluated by ChatGPT, Claude, Gemini, and DeepSeek using identical locked prompts in separate conversations without memory retention or response regeneration. Recommendations were compared with the consultant-led MDT clinical reference standard. Primary outcome was MDT agreement; secondary outcomes included Cohen's κ, domain-specific agreement, assessment-level inter-model consensus, hallucinations, unsafe recommendations, and longitudinal performance using mixed-effects regression with generalized estimating equation and bootstrap sensitivity analyses.ResultsA total of 9592 independent LLM evaluations were analyzed. Overall agreement was highest for Gemini (93.2%; κ = 0.87), followed by Claude (92.9%; κ = 0.86), ChatGPT (91.8%; κ = 0.84), and DeepSeek (88.8%; κ = 0.79). Complete inter-model consensus occurred in 1834 assessments (76.5%), majority consensus in 352 (14.7%), split consensus in 154 (6.4%), and complete disagreement in 58 (2.4%). Of 799 recommendations discordant with the MDT reference standard, 605 met the prespecified criteria for formal adjudication. An additional 95 recommendations were randomly selected from those not meeting the formal criteria, yielding 700 recommendations for blinded expert adjudication. Confirmed safety events were uncommon (2.6-6.0% of evaluations).ConclusionsFrontier LLMs demonstrated high concordance with consultant-led MDT recommendations under standardized prompting conditions. Greater assessment-level inter-model consensus was associated with higher MDT concordance. These findings support further prospective evaluation of LLMs as supervised adjunctive perioperative decision-support tools but do not establish clinical correctness, outcome benefit, or suitability for autonomous decision-making.

The American Surgeon
Imperial College London (GB)
Good health and well-being
Artificial Intelligence in Healthcare and Education
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon