From image to report: Fully AI-generated radiology reports using visual LLMs — A feasibility study on glioma monitoring

Purpose: This study aimed to evaluate the capability of two state-of-the-art visual large language models (vLLMs), GPT-4.o and Sonnet 3.5, to autonomously generate radiological reports by integrating medical history in text form with visual input from longitudinal multisequence MRI data from patients with glioblastoma.The goal was to assess the accuracy and quality of these AI-generated radiology reports (aiRR).J o u r n a l P r e -p r o o f Methods: We included 38 patients with glioblastoma from two academic centers, each with two MRI scans at different time points.The vLLMs generated aiRR based on visual (MRI) and text (medical history) input.These reports were evaluated by three expert neuroradiologists using objective (modality identification, body region, anatomical region accuracy, pathology detection, size estimation, and adherence to the RANO 2.0 criteria) and subjective criteria (report quality, measurement accuracy, and sequence interpretation).Results: Both GPT-4.o and Sonnet 3.5 demonstrated the ability to correctly identify imaging modalities and body regions.However, Sonnet 3.5 outperformed GPT-4.o in key areas, achieving higher accuracy in anatomical region identification (100% vs. 89.47%;p<0.05) and pathology detection (76.32% vs. 55.26%;p=0.06).Both models struggled with lesion size estimation, frequently deviating from expert measurements.Subjective evaluations revealed that Sonnet 3.5 consistently received higher ratings in most evaluation categories e.g.report quality and sequence interpretation.Conclusion: vLLMs demonstrate potential to support radiology workflows by generating reports based on clinical history and imaging data.However, significant limitations in lesion size estimation and pathology detection remain, highlighting the need for further optimization and validation of such models for radiological purposes before clinical application can be considered.