Original Abstracts from the 2025 European Meeting of ISMPP

Objective: To compare and assess 1000-word, text-only plain language summaries (PLS), as created by different generative AI platforms and medical writers.Research design and methods: PLS were created for two different published articles using four generative AI platforms and seven medical writers, self-described as having experience or training in writing for a broad audience.AI platforms and writers received the same guidance and instructions to generate the PLS.PLS underwent initial technical assessment including comparison to the brief, accuracy, reading level, and linguistic analysis.PLS were then blind-evaluated and rated using a Likert scale for understandability, ease of reading, content inclusion, and utility.Reviewers gave an overall rating out of 10 and stated if they thought the PLS were AI-generated.Six PLS were also selected for blinded evaluation by a wider stakeholder panel.Results: All PLS met the brief with respect to length and content.Readability scores and lexical complexity assessments showed little range, with no correlation with human reviewer assessments of readability or ease of understanding.Overall, AI-generated PLS were rated higher on average than human-written PLS across most categories.Ratings for human-written PLS were more variable, with both the lowest-and highest-scoring PLS overall developed by human writers.Reviewers were only 31% accurate in identifying whether PLS were created by AI or human writers.Conclusions: AI-generated PLS have improved significantly since the previous year's analysis 1 .While the highest overall scoring PLS were from PLS-trained writers, AI-generated PLS were generally well-received and scored consistently well, with reviewers unable to distinguish them from human-written ones.