1213 Automatically Translating Lumbar Spine MRI Reports into Patient-Facing Text: A Pilot Analysis of GPT-4

INTRODUCTION: Radiologist interpretations of spine imaging are written for professionals, but a patient’s understanding is crucial to the shared decision-making process. Report summarization by a surgeon is time-intensive. METHODS: Preoperative spine imaging reports from 2022 for patients with degenerative lumbar spondylolisthesis were deidentified and input into the GPT-4 API with the prompt: “Please translate a radiology report into plain language that is easy to understand”. Word counts and Flesch-Kincaid reading-levels were compared. A spine surgeon reviewed all translated reports for inaccuracies and omissions, including major criteria (nerve compression, signs of instability, failure to describe stenosis severity, and failure to describe the location of stenosis) and minor criteria (presence of mild/moderate degenerative changes and mild/moderate canal narrowing). A 1-5 Likert scale was also utilized. Reports with a Likert of 1 were reinputted into GPT with a modified prompt that included five randomly-selected example reports with a Likert of 5 to produce an updated translation. Likert scores and error counts were compared. RESULTS: Of 95 reports, the average word count was 412±173 with a mode reading-level of 12th grade (29%). The GPT translations had a similar word count (400±73, p=0.5220) with a lower reading-level distribution. Six (6%) had a major inaccuracy while 55 (58%) had a major omission. The updated translations for low-performing reports had an improved Likert score (3±1.29 versus 1±0, p=0.0474) and reduced major omissions (6±4.28 versus 15.6±3.19, p=0.0048). CONCLUSIONS: GPT-4 improved the readability of preoperative spine MRI reports. While summarizations rarely included inaccuracies, omissions were more common, highlighting GPT’s lack of domain-specific information. However, when given appropriate example reports, GPT improved its outputs considerably, indicating the potential for LLMs to be continually trained after deployment.