Residual U-Net and Tversky Loss for Multi-Class Anatomical Segmentation in Chest X-Ray Images

This study presents a deep learning based method for the simultaneous segmentation of five anatomical structures in chest X-ray images, namely the left lung, right lung, heart, left clavicle, and right clavicle, using the Japanese Society of Radiological Technology (JSRT) dataset. In the initial configuration, a baseline U-Net model trained with the Cross-Entropy loss achieved low validation loss values; however, the regional overlap metrics did not reach satisfactory levels, and noticeable performance degradation was observed particularly on small anatomical structures. To systematically examine the effects of residual connections and the Tversky loss function, four model configurations were evaluated: (i) U-Net with Cross-Entropy, (ii) U-Net with Tversky, (iii) Residual U-Net with Cross-Entropy, and (iv) Residual U-Net with Tversky. The results show that the Tversky loss alone increased the Dice score from 0.296 to 0.548, while residual connections increased it to 0.444. The configuration combining both components achieved the highest performance, reaching an average Dice score of 0.826 and a Jaccard score of 0.704 on the test set. Dice values reached the range of 0.86–0.88 for the lung regions, while scores of 0.696 and 0.817 were obtained for the heart and right clavicle, respectively. In contrast, low performance was observed for left clavicle segmentation across all configurations (maximum Dice: 0.108), which is attributed to class imbalance, anatomical variation, and low contrast. Overall, the findings indicate that pixel-wise Cross-Entropy loss does not directly optimize regional overlap, whereas the combined use of residual learning and the Tversky loss provides a more stable and accurate solution for multi-class chest anatomy segmentation.