Semantic segmentation of TB in chest x-rays: a new dataset and generalization evaluation

According to the 2023 World Health Organization report, an estimated 7.5 million people were diagnosed with tuberculosis (TB) in 2022. TB triaging is often performed using chest X-rays (CXRs), with significant efforts invested in automating this task using deep learning. A key concern with algorithms that output image-level labels, in our context TB/not-TB, is that they do not provide an explicit explanation with respect to how the output was obtained, limiting the ability of user oversight. Semantic segmentation of TB lesions can enable human supervision as part of the diagnosis process. This work presents a new dataset, TB-Portals SIFT, which enables semantic segmentation of TB lesions in CXRs (6,328 images with 10,435 pseudo-label lesion instances). Using this data, ten semantic segmentation models from the UNet and YOLOv8-seg architectures were evaluated in a five-fold cross validation study. The best performing segmentation models from each architecture, nnUNet(ResEnc XL) and YOLOv8m-seg and their ensemble were then evaluated for generalization on related classification and object detection tasks. Additionally, several binary DenseNet121 classifiers were trained, and their classification generalization performance was compared to that of the semantic segmentation-based classifier. Results show that the segmentation-based approach achieved better generalizability than the DenseNet121 classifiers and that the ensemble of the models from the two architectures was the most stable, closely matching or exceeding the performance of all other models across the tasks of segmentation, classification, and object detection. The dataset is publicly available from the NIAID TB Portals program after signing a data usage agreement which is available from https://tbportals.niaid.nih.gov/download-data.