Self-reported comprehension of large language model-generated summaries of lung cancer screening reports: a vignette survey

Abstract Background Patients access radiology reports without delay, which can cause anxiety and misunderstanding. While large language models (LLMs) can generate patient-friendly summaries (PS) to mitigate this, their potential to address literacy-based disparities remains unquantified. Purpose To measure the effect of LLM-generated PS on the objective comprehension and subjective experiences of individuals reading lung cancer screening computed tomography reports, and to determine the differential impact with respect to self-rated English and health literacy. Materials and Methods This cross-sectional survey (July 24–28, 2025) used a within-subjects design. Participants from the online research platform Prolific, self-enrolled from the general U.S. population, viewed 3 lung cancer screening reports (negative; negative with complex incidentals; suspicious for malignancy), first in their original format and then with an LLM-generated PS. Objective comprehension, subjective experiences (including anxiety, via a 5-point Likert scale), and hypothetical communication intent were assessed after each viewing. Univariate and multivariate analyses, including demographic subgroup comparisons, were performed. Results A total of 1815 participants (mean age, 46 years ± 16 [SD]; 919 women) who completed the survey were evaluated. The addition of a PS improved objective comprehension (P < .001) and reduced anxiety (P < .001) for all scenarios. This effect was most pronounced for respondents with low self-rated English literacy, who had greater comprehension gain (P < .001) and anxiety reduction (P = .012) than those with high literacy. Individuals with low self-rated health literacy also experienced more anxiety reduction (P < .001). PS also increased the proportion of participants reporting that they would be willing to wait for a scheduled appointment to discuss their results (P < .001). Conclusion LLM-generated PS of lung cancer screening reports increase comprehension and reduce anxiety, most notably among individuals with lower self-rated English and health literacy. If validated in a patient population, they represent a potential tool to improve communication.