Novel deep learning approach combining image reconstruction and diagnostic segmentation

Motivation: Deep-learning-based image reconstruction methods for accelerated magnetic resonance imaging are optimized for global image quality metrics, but lack focus on diagnostically relevant features during training and reconstruction. Goal(s): A novel approach combining reconstruction and segmentation during training is investigated, incorporating feedback on clinically relevant features for reconstruction. Approach: Pretrained reconstruction (E2E-VN) and segmentation models (nnUNet) are connected. The reconstruction model is trained with a weighted combination of reconstruction and segmentation loss. Training and evaluation are performed on fastMRI+ data. Results: The proposed method resulted in improved image quality of reconstructed images at 8x acceleration compared to baseline E2E-VN, along-with improved downstream segmentation. Impact: Training deep-learning-based image reconstruction methods for accelerated MRI with additional feedback on diagnostic content improves image quality in the overall image and the region of interest, and subsequently the diagnostic utility of reconstructed images.