Auto-Rad: End-to-End Report Generation from Lumber Spine MRI Using Vision–Language Model
<b>Background:</b> Lumbar spinal stenosis (LSS) is a major cause of chronic lower back and leg pain, and is traditionally diagnosed through labor-intensive analysis of magnetic resonance imaging (MRI) scans by radiologists. This study aims to streamline the diagnostic process by developing an automated radiology report generation (ARRG) system using a vision-language (VL) model. <b>Methods:</b> We utilized a Generative Image-to-Text (GIT) model, originally designed for visual question answering (VQA) and image captioning. The model was fine-tuned to generate diagnostic reports directly from lumbar spine MRI scans using a modest set of annotated data. Additionally, GPT-4 was used to convert semistructured text into coherent paragraphs for better comprehension by the GIT model. <b>Results:</b> The model effectively generated semantically accurate and grammatically coherent reports. The performance was evaluated using METEOR (0.37), BERTScore (0.886), and ROUGE-L (0.3), indicating its potential to produce clinically relevant content. <b>Conclusions:</b> This study highlights the feasibility of using vision-language models to automate report generation from medical imaging, potentially reducing the diagnostic workload for radiologists.
Authors
- Kazi Ashraf Moinuddin (ORCID: https://orcid.org/0000-0001-8742-2019)
- Paul Park (ORCID: https://orcid.org/0000-0002-0375-396X)
- Felix Havugimana (ORCID: https://orcid.org/0000-0001-9208-2549)
- Mohammed Yeasin (ORCID: https://orcid.org/0000-0003-0579-3030)
- Lijia Wang (ORCID: https://orcid.org/0000-0002-1907-171X)
Institutions
- University of Tennessee Health Science Center (US)
- University of Memphis (US)
Publication Details
- Journal
- Journal of Clinical Medicine
- Published
- 2024-11-23
- DOI
- https://doi.org/10.3390/jcm13237092
- Citations
- 6
- Primary Topic
- Multimodal Machine Learning Applications
- Type
- article
- Field-Weighted Citation Impact
- 1.50