SALT: Introducing a framework for hierarchical segmentations in medical imaging using label trees

Traditional segmentation networks treat anatomical structures as isolated elements, often neglecting their hierarchical relationships. This study introduces Softmax for Arbitrary Label Trees (SALT), a novel method that leverages these hierarchical connections to improve segmentation efficiency and interpretability. SALT is a novel activation function that extends the softmax to arbitrary hierarchical label trees by modeling conditional probabilities along parent-child relations in imaging data. This enables anatomical hierarchies in CT imaging to be represented naturally, allowing segmentations to propagate from broader regions to detailed structures. Using the SAROS dataset from The Cancer Imaging Archive (TCIA), which comprises 900 body segmentations from 883 patients and was enhanced by TotalSegmentator to include 113 anatomical labels, the model was trained on 600 scans, with validation and testing on 150. Dice scores across SAROS, CT-ORG, FLARE22, LCTSC, LUNA16, and WORD datasets measured performance, with 95% confidence intervals (CI) from 1000 bootstrapping rounds. SALT achieved top performance on the LUNA16 and SAROS datasets, with Dice scores of 0.93 (95% CI: 0.919-0.938) and 0.929 (95% CI: 0.924-0.933). Reliable accuracy was also observed in CT-ORG (0.891, 95% CI: 0.869-0.906), FLARE22 (0.849, 95% CI: 0.844-0.854), LCTSC (0.908, 95% CI: 0.902-0.914), and WORD (0.844, 95% CI: 0.839-0.85). SALT can segment a 1000-slice CT in 35 s. By leveraging hierarchical body structures, SALT enables rapid whole-body segmentation, processing CT scans in 35 s on average. This capability supports its integration into clinical workflows, enhancing efficiency in automated full-body segmentation and contributing to improved diagnostic workflows and patient care.