GPT-4o-Powered Preanesthetic AI: Development and Validation Study.

Background: Accurate preanesthetic assessment is essential for perioperative risk stratification, but conventional tools such as the American Society of Anesthesiologists (ASA) physical status classification and postoperative nausea and vomiting (PONV) risk scores may be affected by subjective judgment, incomplete documentation, and fragmented clinical data. Large language models may support preanesthetic assessment by integrating structured and unstructured clinical information. Objective: This study aimed to develop and retrospectively validate a GPT-4o (OpenAI)-powered AI system for preanesthetic assessment. The primary validation focus was the agreement between AI-generated and clinician-assigned ASA physical status classifications. PONV risk stratification was evaluated as an additional clinically relevant performance outcome. A secondary objective was to assess the incremental contribution of National Health Insurance (NHI) cloud data to model performance. Methods: This single-center retrospective validation study included 600 adult surgical patients randomly sampled from 4404 eligible inpatient surgical patients between January and May 2025. The system processed structured and unstructured electronic health record data, with and without NHI cloud data, to generate preanesthetic assessment outputs. ASA agreement was assessed using Cohen κ with 95% CIs. PONV prediction was evaluated using sensitivity, specificity, overall accuracy, positive predictive value, negative predictive value, confusion matrix analysis, the 2-sided Fisher-Freeman-Halton exact test, and Cramér V. Results: <.001, Cramér V=0.531). Conclusions: The GPT-4o-powered preanesthetic assessment system demonstrated feasibility and high agreement with clinician-assigned ASA classifications when NHI cloud data were incorporated. For PONV risk stratification, the system showed high specificity and a high negative predictive value but modest sensitivity, indicating that it identified a small high-risk subgroup with a high observed PONV incidence but did not capture all patients who developed PONV. These findings support the use of large language model-based decision-support tools as adjuncts to, rather than replacements for, anesthesiologist judgment.