Expert appraisal of AI-generated recommendations in cardiovascular surgical decision-making: a multisurgeon study

Large language models are increasingly considered for clinical decision support, but their acceptability in complex surgical decisions remains uncertain. This study evaluated cardiovascular surgeons’ perceptions of the clinical appropriateness of recommendations generated by GPT-5.2 and examined whether ratings differed according to surgeon experience and scenario type. Fifty cardiovascular surgical scenarios, including approximately 20% predefined guideline-ambiguous (“grey-zone”) cases, were processed using a standardized instruction. Fourteen practicing cardiovascular surgeons independently rated each recommendation on an anchored five-point Likert scale, producing 700 evaluations. Ratings were analyzed using cross-classified linear mixed-effects models with surgeon and scenario as random effects. The overall mean rating was 4.29 ± 0.90, and 85.9% of evaluations were classified as clinically appropriate. Early- and Mid-Career surgeons gave comparable ratings, whereas Senior surgeons assigned lower scores than Mid-Career surgeons (mean difference − 0.27, p = 0.001). Grey-zone scenarios also received lower ratings (mean difference − 0.19, p = 0.023). These findings indicate that GPT-5.2-generated recommendations were generally perceived as clinically appropriate, particularly in guideline-based scenarios, while differences associated with seniority and clinical ambiguity underscore the continued importance of expert judgment.