Comprehension of an AI-generated discharge letter versus a traditional discharge letter: two controlled quasi-experimental parallel-group studies

Abstract Background Discharge letters are essential for continuity of care, yet patients often misunderstand their content. Poor discharge communication is associated with medication errors, inappropriate healthcare use, and hospital readmissions. Artificial intelligence (AI) language models may improve discharge documentation clarity and accessibility. This study aimed to compare a GPT-4-generated, accessibility-optimised discharge letter with a conventional discharge letter regarding comprehension of key discharge information, and to evaluate satisfaction and identify predictors of comprehension. Methods Two controlled quasi-experimental parallel-group studies were conducted. Study 1 included adults recruited online (n = 341 analysed), while Study 2 involved medical and nursing students at the University of Turin (n = 791 analysed). Allocation was based on age parity. The primary outcome was a structured comprehension score assessing diagnosis, treatment, investigations, and follow-up instructions. Secondary outcomes included readability, clarity, perceived comprehension, organisation, and satisfaction. Health literacy (HL) was assessed using the HLS-EU-Q6. Multiple linear regression analyses identified predictors of comprehension. Results Participants receiving the AI-generated discharge letter achieved significantly higher comprehension scores in both studies (Study 1: median 4 vs. 2; Study 2: median 4 vs. 3; both P < 0.001). Differences remained significant after adjustment for sociodemographic variables, HL, and healthcare background. AI-generated letters also improved all secondary outcomes (all P < 0.001). No significant improvement was observed for identification of follow-up appointment dates. Higher comprehension was associated with healthcare background, higher education, sufficient HL, and better perceived health status. Conclusion AI-generated discharge letters significantly improved measured comprehension and satisfaction in both general and higher-literacy populations; validation using real clinical letters and diverse patients is required.