Introducing cosine similarity into the loss function: application to 3D vertebra segmentation

Segmentation of medical images is essential for diagnostics and treatment planning, with vertebra segmentation playing a crucial role in evaluating spinal diseases and vertebral fractures. Precise segmentation in three-dimensional (3D) computed tomography (CT) images of the spine enables a better assessment of spine conditions, which is important for planning surgical interventions, guiding surgical procedures, and for follow-up evaluation. Modern segmentation frameworks are based on deep learning, where loss functions are an important element that influences their convergence. In this study, we propose a novel loss function, i.e. the cosine similarity cross-entropy loss function, that is based on the recently introduced cosine similarity, and apply it for 3D vertebra segmentation. We evaluate the performance of the proposed loss function on a convolutional- and a transformer-based network, both individually trained and tested on the publicly available VerSe2020 dataset, containing 312 CT spine images, and on an in-house dataset based on the publicly available CTSpine1K dataset, containing 1,218 CT spine images. The results show that the cosine similarity cross-entropy loss function outperforms the conventional Dice cross-entropy loss function by an average of 3.4%pt (percentage points) in terms of the Dice similarity coefficient and an average of 2.60mm in terms of the 95th percentile Hausdorff distance between the ground truth and predicted vertebra segmentations.