Supplemental Information: Readability Assessment and Comparison of Large Language Model-Generated Summaries of Trial Descriptions on ClinicalTrials.gov

This study evaluated the readability of ClinicalTrials.gov trial information using traditional readability measures (TRMs) and compared it to summaries generated by large language models (LLMs), specifically ChatGPT and a fine-tuned BART-Large-CNN (FBLC). The study involved: 1) assessing required reading levels (RRL) with TRMs, 2) generating LLM-based summaries for a sample, and 3) evaluating summary quality through content and ranking scores averaged by two reviewers. Results showed that original summaries scored above the recommended readability level and did not improve over time. ChatGPT-generated summaries, however, had significantly lower RRLs and higher quality scores compared to original and FBLC summaries. High RRLs in ClinicalTrials.gov summaries may hinder participant recruitment and consent. ChatGPT shows promise for creating readable, high-quality summaries, and future research will evaluate ChatGPT-generated summaries and improve readability assessment metrics.