INNV-21. Initial experience with generative AI for clinical summarization in glioma MR interpretation

Abstract BACKGROUND Patients with primary brain tumors often have very complex histories, but an accurate understanding of their care timeline is critical to provide the best care possible. Extracting pertinent facts from the medical record can be time consuming and frustrating. We sought to apply generative AI large language models (LLM) to this problem in order to supply providers with concise, accurate, and focused data including graphs and tables. METHODS After evaluating multiple LLMs, GEMINI 2.5 Pro was selected. Prompt engineering was iteratively refined across a range of patients to generate concise outputs using tables, graphs, and prose to describe timing of chemoradiation, tumor markers, procedures, and trends in MRI results. The model inferred BT-RADS scores from the MRI report prose. Using a 5-point Likert scale, neuroradiologists assessed the AI-output both prospectively and retrospectively comparing it to the medical record. LLM’s compared AI summaries to source material. RESULTS Six neuroradiologists each ranked 25 prospective patient outputs as accurate (82%/16%/1%), useful (94%/6%/0%), trustworthy (86%/13%/1%), and satisfactory (86%/10%/1%) for Likert ratings of highly agree/agree/neutral, respectively. No responses were disagree or highly disagree. Interobserver agreement was high (Gwent AC2 > 0.95). The LLM evaluation of referenced portions of the record determined > 85% of summary statements were supported by source documents. While users estimated time savings of a 1-10 minutes per case, they felt the greater benefit was having more organized data presented efficiently. CONCLUSIONS AI summaries of the medical record of complex patients with primary brain tumors give providers an accurate, well-organized display of the pertinent facts necessary for high quality care and save time in chart review. These summaries, which are generated early each morning, have been found to be helpful and are now being incorporated into our clinical practice and multidisciplinary tumor board.