Model-EY: Enhanced Yield Transformer Based Turkish Chest X-Ray Medical Report Generation

Writing medical reports from chest X-ray images in traditional methods is a time-consuming and critical process that requires specialized medical practitioners. In addition, autonomous report generation in the Turkish language is an area that has not been studied much yet and is open to development. In this study, we aim to generate medical reports in Turkish using deep learning methods from chest X-ray images. In this study, an encoder-decoder based hybrid deep learning model called Model-EY was developed. Vision Transformer architecture is used in the encoder part of the model, and GPT architecture, which is fine-tuned specifically for the Turkish language, is used in the decoder part. With the permission of the ethics committee, a new dataset was created using image-report pairs obtained from Elazıg Fethi Sekin City Hospital and Indiana University Chest X-Ray dataset and experiments were conducted on this new dataset. In the tests conducted within the scope of the study, scores of 0.6307, 0.5283, 0.4371 and 0.3690 were obtained in Bleu-1, Bleu-2, Bleu-3 and Bleu-4 word overlap evaluation metrics, respectively. The results produced by the encoder-decoder architecture were also checked by an expert medical doctor in our research team and it was stated that the results can help doctors in decision-making. The proposed model can reduce the workload of doctors and prevent possible human errors during diagnosis.