Contextual Reinforcement Learning for Unsupervised Deformable Multimodal Medical Images Registration

Multimodal deformable image registration refers to the process of finding the spatial correspondence between pairs of images with multimodal and mapping them onto the same coordinate system. Most of the deep learning-based registration methods are one-shot registration, which is difficult to handle images with significant deformations or displacements. Reinforcement learning can handle these challenges by viewing registration as a strategic decision-making process which is a step-by-step registration. However, it faces challenges with high-dimensional and continuous deformation fields. To overcome this, we introduce a planner network that maps high-dimensional input state to low-dimensional plan, guiding the actor to generate continuous actions. In order to handle complex multi-modal registration, we propose a multi-frame plan module which encourages artificial agent to explicitly utilize the redundant states in the registration process and learn more accurate registration actions from the generated state frames. To facilitate the training and convergence of the model, we define an unsupervised reward function and incorporate spectral normalization layers. The entire framework is a fully unsupervised registration framework and training in an end-to-end manner. We evaluated our method on publicly available T1w and T2w brain datasets, and the results indicate that our method has excellent deformable registration capability for multimodal images.