Perioperative Risk Stratification with AI-Powered Chatbots: A Systematic Review and Meta-Analysis

Background: Chatbots are becoming increasingly valuable in clinical settings, offering rapid access to medical information, aiding documentation, and improving perioperative patient education. Their adaptability makes them promising tools for personalized perioperative risk stratification (PRS) and anesthesia planning, but their definitive role remains uncertain. We aimed to evaluate chatbot performance in PRS compared to standard clinical judgment and to assess the certainty of the evidence supporting their use. Methods: This systematic review (PROSPERO ID: CRD42025642357) followed PRISMA extended and PRISMA-S guidelines. The population was defined according to the PICO framework: we included adult surgical patients undergoing anesthesia assessment (P), evaluated with LLM-based chatbots for perioperative risk stratification and anesthesia planning (I), compared with traditional clinician assessment (C), and extracted performance metrics (O). Comprehensive searches of PubMed, MEDLINE, Scopus, Embase, Google Scholar, Open Gray, ClinicalTrials.gov, WHO ICTRP, and Cochrane Library Central were conducted through January 2026. Risk of bias and study quality were assessed using PROBAST-AI, RoB-2, and ROBINS-I. Certainty of the evidence was assessed using GRADE system. A random-effects meta-analysis of pooled chatbot accuracy was performed, with subgroup analyses by ASA status and perioperative risk stratification. A sensitivity analysis was performed with a leave-one-out exclusion test. Results: Eleven studies published between 2023 and January 2026 were included (N = 227,059 patients). Five prospective cohorts, two large retrospective cohorts, one randomized non-inferiority trial, and three non-clinical or mixed-methods studies were found. Meta-analysis showed that the pooled accuracy of LLM-based chatbots for AI–clinician concordance in perioperative risk stratification and ASA classification was 0.90 [95% CI: 0.42–0.99; 95% prediction interval 0.03–1.00]. Subgroup analyses indicated that the ASA status prediction subgroup reached a pooled accuracy of 0.91 (95% CI: 0.46 to 0.99), whereas the exploratory perioperative risk stratification subgroup showed an accuracy of 0.73 (95% CI: 0.10 to 0.98). Performance decreased with increasing patient complexity. Evidence is limited by small sample sizes, extreme sample size skew toward a single center, geographic bias, inconsistent outcome definitions and performance metrics, and incomplete reporting of adverse events. Most studies lacked prospective trial registration or robust control for confounding, and publication bias cannot be excluded. Conclusions: LLM-based chatbots show promising performance in routine perioperative risk stratification but remain unreliable in complex cases, with potential safety concerns. Given the overall very low GRADE certainty of evidence, these tools should be used as clinician-supervised decision support aids for routine ASA assessment, and should not be relied upon for autonomous use in complex cases or for general perioperative risk stratification. Other: This research received no external funding. PROSPERO ID: CRD42025642357.