Improved Medical Image Captioning Using Vision Transformers and Graph Networks

Image captioning intersects Computer Vision (CV) and Natural Language Processing (NLP), aiming to generate fitting captions for specific images. This area has many applications, from recommendation systems to medical report generation and social media. One challenge within medical report generation is creating captions with detailed information, such as the scan location and the relationships between image objects and clinical findings. Though research exists in this field, there is limited comprehensive coverage of its various challenges. This paper focuses on medical image captioning for chest X-rays. It identifies image concepts separately and generates captions using graph networks and deep learning architectures. This method broadens the potential applications of medical report generation and offers a more intuitive image captioning framework.