An Experimentation for generating Text-based Medical Report from Chest X-Ray images using Attention based Deep Learning Models

This study investigates the efficacy of attention-based deep learning models for generating text-based medical reports from chest X-ray images. Four distinct models were developed and evaluated: a basic encoder-decoder model (Model 1), an encoder-decoder architecture using an attention mechanism (Model 2), a model incorporating spatial feature preservation (Model 3), and a model with a bidirectional GRU in the decoder (Model 4). We trained and evaluated these models using the Indiana University Chest X-ray dataset (Open-i), employing the BLEU score as the primary performance metric. Model 1, using a greedy search decoding strategy, achieved an average BLEU score of 0.619. Incorporating an attention mechanism in Model 2 resulted in a modest improvement, reaching a BLEU score of 0.667 with beam search decoding. Model 3, preserving spatial information during feature extraction, further enhanced performance, achieving a BLEU score of 0.718. Finally, Model 4, integrating a bidirectional GRU, yielded the highest performance with a BLEU score of 0.745. Our results highlight the significant impact of attention mechanisms and spatial feature preservation in generating more accurate and detailed medical reports. The study highlights the potential of deep learning models for automating medical report generation, paving the way for further research and development in this domain.