BioVLM-T: A temporal framework for radiology report generation using pre-trained vision language foundational models

In this study, we propose BioVLF-T, a novel automatic radiology report generation framework built on a bio-vision language foundational model (VLF) with a temporal framework. BioVLF-T enhances the contextual understanding of the vision-language model by leveraging historical patient data, incorporating previous chest x-rays and their corresponding radiology reports. We fine-tuned the BLIP-VQA, a vision-language foundational model, within this framework, integrating a CNN-based instructor model pre-trained on 14 pathological labels from CheXbert to accurately capture relevant pathologies. By sequentially integrating historical data, the model simulates a process akin to that of a radiologist, resulting in more precise and contextually relevant report generation. Our experimental results demonstrate significant performance enhancements, with the BioVLF-T framework showing a 120.47% improvement in BLEU scores and a 1069.50% increase in CIDEr scores over the conventional fine-tuning method. These improvements highlight the effectiveness of temporal data integration in radiology report generation. Additionally, we acknowledge the limitations of traditional metrics such as CIDEr in capturing the nuances of language and, in the future, will focus on scaling the model with larger datasets and exploring advanced evaluation methods, including LLM-based scores, to enhance prediction quality further.