Is segmentation performance of deep-learning models affected by cancer type? A performance analysis on PET/CT

In cancer care, positron emission tomography/computed tomography (PET/CT) is used across a variety of cancer types. Though automatic tumor segmentation in PET/CT is improving with advanced deep learning (DL), their performance across different cancer types is under-explored. Therefore, we evaluated two state-of-the-art DL models using a publicly available PET/CT dataset that includes three cancer types: lung cancer, lymphoma, and melanoma, along with normal controls (1014 studies overall). This dataset’s uniform image acquisition protocol (including scanner) mitigates domainshift and allows for a reliable evaluation. The models assessed, nnU-Net and Segment Anything Model (SAM), were prompted with bounding-boxes based on nnU-Net segmentation (nnUNet-SAM). The evaluations were performed using five-fold cross-validation investigating: (a) patient-based analysis (all tumor sites together) and (b) lesion-based analysis (isolated tumor sites) using the Dice coefficient (DC) as the evaluation metric. The Kruskal-Wallis H test and test of proportions were applied to compare performance among the cancer types. The patient-based analysis revealed variations among lung cancer (N = 168), lymphoma (N = 145), and melanoma (N = 188) using average DC (0.73, 0.73, 0.63, p = 0.002, using nnU-net which performed better). The lesion-based analysis indicated significant variation (p < 10-5 ) among the cancer types using average lesion-based DC. This variation was lower considering the highest lesion-based DC. The largest lesion contributed more frequently to the highest lesion-based DC in lung cancer (71% – 90%) compared to lymphoma (63% – 88%) and melanoma (57% – 76%). The findings underscore the need for a personalized tumor segmentation approach and suggest taking caution for automated tumor analysis across cancer types using PET/CT.