LLM-Generated Visual Summaries of Cochrane Plain-Language Summaries: A Pilot Expert-Rated Study and Implications for Research Communication

Background: Plain-language summaries (PLSs) improve accessibility of medical research for patients but remain predominantly text-based. Large language models (LLMs) can now generate images from text. We carried out the present exploratory study to evaluate whether LLMs can generate images that demonstrate technical accuracy and theoretical visual usability when derived from PLS content. Methods: In this cross-sectional pilot study, two PLS were randomly selected from each of 37 Cochrane Library themes. Three LLMs, ChatGPT-5.2, Google Gemini 3 Pro, and Google Notebook, generated one image per PLS, yielding 222 images. Two blinded assessors evaluated images using a preliminary, internally expert-validated tool that measured technical accuracy and completeness, visual usability, and hallucination presence. Inter-LLM comparisons were assessed using linear mixed effects model. Results: Gemini outperformed ChatGPT and Notebook across all domains (p < 0.001). Hallucinations occurred exclusively in ChatGPT-generated images (29.73%). Gemini demonstrated the least intra-thematic variability, whereas ChatGPT showed the highest. No significant interaction was found between LLM type and Cochrane theme. Sensitivity analyses, including alternative weighting schemes and leave-one-theme-out analyses, confirmed robust model rankings. Conclusions: In this single-prompt expert-rated pilot study, Google Gemini 3 Pro reliably generated accurate, hallucination-free visual summaries from PLS. These exploratory findings support further patient-centered validation of LLM-generated images as complements to text-based patient education materials.