Development of Methods for the Registration of Multimodal Medical Images Using Deep Learning

In the field of medical image processing, multimodal medical image registration is an essential technique due to its fundamental role for integrating complementary information from diverse imaging modalities which is used in various clinical applications, e.g. in the diagnosis and planning and performing of treatments, e.g. for renal and oligometastatic diseases. Innovations in medical image registration increase the accuracy of image fusion, and thus improve patient care. Therefore, the aim of this thesis is to improve and develop new deep learning-based image registration methods for three-dimensional multimodal medical images. In the first part, 20 different published neural networks were implemented for affine medical image registration and their generalisability to new data was evaluated using two multimodal medical datasets and an in-house developed neural network as benchmark. Only six and nine networks, respectively, significantly improved the pre-registration Dice coefficient and therefore generalized well to the new datasets. This shows that further research is needed to develop medical image registration methods that can be used for a wider range of medical applications. In the second part, a novel multistage neural network for multimodal medical image registration was presented, addressing large rigid deformations typical in medical imaging. The performance of the novel multistage network, which applies rigid, affine and deformable transformations, was compared with that of a multistage network using affine and deformable transformations. Utilising the novel multistage network resulted in high spatial alignment (Dice up to 68.1 ± 24.6) and medically plausible transformations (|J| ≤ 0 :≤ 1.1%) on four multimodal medical datasets. In the third part, neural networks for deformable multimodal medical image registration were developed that apply multiple steps and resolutions in a coarse-to-fine approach. The potential benefits and limitations of using multiple steps and resolutions were investigated using four multimodal medical datasets. The results demonstrated that the multistep networks achieve high structural similarity (NMI up to 0.33 ± 0.02, Dice up to 90.8 ± 3.1) and medically plausible registration results (|J| ≤ 0 :< 0.5%) and utilising more resolutions (up to 4 in this study) results in increased spatial alignment. In the fourth study, a groupwise multiresolution network was presented for the deformable registration of DCE-MR images for the assessment of lung perfusion in patients with congenital diaphragmatic hernia. Experimental results showed that the proposed groupwise network outperformed iterative and deep learning-based pairwise and groupwise registration methods and led to increased homogeneity of the perfusion maps, enhancing the accuracy of medical image analysis. In conclusion, the deep learning-based medical image registration methods developed in this thesis improve the accuracy of medical image registration and may serve as valuable tool for clinical applications of image fusion, such as diagnosis of diseases and treatment planning. By enabling rapid registration, these techniques support real-time decision making and increase the accuracy of interventions, ultimately leading to better patient outcomes.