Multimodal Learning for Lung Segmentation: Enhancing UTE MRI Segmentation with CT Datasets

Ultrashort echo time (UTE) magnetic resonance imaging (MRI) produces high-resolution structural images of the lungs without the ionizing radiation risks associated with computed tomography (CT) imaging. Lung segmentation in UTE is a necessary precursor to biomarker analysis, however, there are challenges associated with limited labeled training data, intensity inhomogeneity, noise, and image gradients. In this work, CT datasets are leveraged to improve the robustness of UTE lung segmentation, given limited labeled training data in the UTE domain. A novel joint training framework is proposed to simultaneously learn structural patterns important for multimodal lung segmentation. Additionally, a learnable downsample layer is proposed to enable training on full 3D CT and MRI images, while maintaining network complexity. Lastly, a false positive penalization loss is proposed to decrease erroneous segmentations associated with noise and gradients in UTE MRI. An ablation study evaluates each component's contribution, with the proposed approach achieving a Dice coefficient of 0.97 and a surface distance of 1.05 mm.