Higher Final-Response Under-Triage with Multi-Turn Patient-Pressure Prompts in Consumer-Facing ChatGPT: A Controlled Postoperative General Surgery Study
Background/Objectives: Postoperative care-seeking advice from consumer chatbots may change when users resist medical evaluation. We compared final recommendations after scripted patient-pressure versus neutral follow-ups. Methods: Forty-eight clinician-authored scenarios across six procedure groups were tested in both conditions and three fresh-chat runs (288 conversations; 1440 responses). Two surgeons established G0–G3 reference grades and coded all 288 Turn-5 responses. Study-defined under-triage was assessed among G1–G3 scenarios (216 responses). A scenario random-intercept logistic model was primary, with paired and marginal analyses. Following peer review, an independent general surgeon characterized all 216 eligible Turn-5 responses post hoc. Results: Final-response agreement was 277/288 (96.2%; Cohen’s kappa, 0.945). Study-defined under-triage occurred in 10/108 Neutral responses (9.3%) versus 25/108 Pressure responses (23.1%): absolute risk difference, 13.9 percentage points (95% scenario-cluster bootstrap CI, 5.6–23.1); risk ratio, 2.50 (95% scenario-cluster bootstrap CI, 1.38–8.00). The conditional mixed-effects odds ratio was 5.24 (95% CI, 1.84–14.96), versus a marginal GEE odds ratio of 2.95 (95% CI, 1.33–6.53). In 108 matched pairs, Pressure alone produced under-triage in 18 pairs and Neutral alone in three (McNemar p = 0.00149). Independent characterization found explicit rejection of delay/home-only care in 21/25 (84%) Pressure under-triage responses, including all 12 G3→G2 events. Conclusions: Patient-pressure prompts were associated with more final study-defined under-triage, often reflecting less specific or less directive care pathways rather than explicit acceptance of delay. No patient outcomes were observed.
Authors
- Serdar Yormaz (ORCID: https://orcid.org/0000-0002-6273-3643)
- Fatih Türkoğlu (ORCID: https://orcid.org/0000-0001-5128-4419)
Institutions
- Selçuk University (TR)
Publication Details
- Journal
- Diagnostics
- Published
- 2026-11-16
- DOI
- https://doi.org/10.3390/diagnostics16223811
- Primary Topic
- Artificial Intelligence in Healthcare and Education
- Type
- article
- Field-Weighted Citation Impact
- 0.00