Artificial Intelligence in Patient Communication: Performance of GPT-3.5 and GPT-4 in Coronary Bypass Surgery

Objective: This study aims to evaluate the ability of GPT-3.5 and GPT-4 to provide accurate, comprehensible, and clinically relevant responses to common patient questions about coronary bypass surgery. Method: A cross-sectional study was conducted at Ankara Yıldırım Beyazıt University Bilkent City Hospital with 80 cardiovascular surgery specialists. Participants rated the responses of GPT-3.5 and GPT-4 to 10 common patient questions about coronary bypass surgery based on four criteria: accuracy, understandability, clinical appropriateness, and overall evaluation. Statistical analysis included independent t-tests, Cronbach’s Alpha reliability analysis, and Cohen’s d effect size calculation. Results: GPT-4 significantly outperformed GPT-3.5 across all metrics. The mean scores for GPT-4 were higher in accuracy (3.02 vs. 1.77), understandability (2.99 vs. 1.81), clinical appropriateness (2.96 vs. 1.78), and overall evaluation (2.98 vs. 1.77) (p