COVSeg-VLM: vision-language model for reliable segmenting COVID-19 infections in chest x-ray images

Automated lung infected region delineation in X-ray images is essential for accurately diagnosing COVID-19 and assessing disease progression. In this paper, we introduced the COVID-19 Infection Segmentation Vision Language Model (COVSeg-VLM) network for infected region segmentation in 2D X-ray images. Although the advancements in vision deep-learning neural networks have significantly enhanced the auto-contouring of normal organs and anatomical structures, accurate delineation of infected region of COVID-19 remains a challenge. Our model adeptly incorporates large language models to extract text-rich features from clinical reports, captures both vision- and text-contextual information for accurate segmentation. We assessed our model using the QaTa-COV19 Dataset, showcasing its superior performance compared to conventional models, especially in difficult cases. The results underscore our network's robustness and precision, validating its efficiency in both typical and complex segmentation scenarios. These improvements herald a new era of reliable and accurate medical image analysis, with promising implications for enhancing clinical outcomes.