Predicting anatomical structures from 2D depth images of patients

The knowledge of anatomical structures’ locations prior to radiological imaging allows for the improvement of MRI and CT examination efficiency, as well as a reduction in the required radiation dose for CT imaging. The objective of this work is to develop a deep learning model for the prediction of 2D projections of bone, organ, muscle and vessel masks from a 2D depth image of a patient, which resembles a depth image taken by time-offlight cameras. To achieve this goal, CT and MRI images are segmented in 3D and projected down to a 2D depth map of the patient surface and 2D projections of corresponding anatomical labels. The models are trained using the TotalSegmentator CT and German National Cohort MRI data and additionally evaluated using the ACRINFLT- Breast data. The model achieves an average symmetric surface distance (mean ± standard deviation) of 6.97 ± 12.52, 4.55 ± 4.91 and 6.39 ± 7.54mm, respectively.