Vision-Language Transformer Framework for Automated Medical Imaging Reporting

Automated medical imaging report generation is an emerging challenge in clinical AI, aiming to bridge visual data interpretation with natural language understanding. While conventional approaches typically rely on Convolutional Neural Networks (CNNs) for visual feature extraction, recent progress in vision transformers has introduced more powerful alternatives for representation learning. In this study, we propose a unified vision-language encoder-decoder framework for generating diagnostic reports across various medical imaging modalities. The framework integrates visual features extracted from state-of-the-art models including Vision Transformer (ViT), CLIP, Transformer-based, and CNN-based architectures, which are then projected into a shared embedding space via a multilayer perceptron (MLP). This facilitates seamless alignment with advanced language generation models. Our experiments are conducted on a small-scale clinical dataset, the IU-Xray dataset, to evaluate the effectiveness of the proposed architecture in low resource settings. The results highlight the potential of transformer based vision-language models, particularly when leveraging domain specific language models like BioBART, for generating coherent and clinically relevant radiology reports.