Super Resolution in Medical Imaging Time-Series data using Neural Networks

Super Resolution (SR) is a critical task in computer vision, particularly in the medical imaging domain, where the spatial and temporal resolution of images is often limited due to acquisition time and hardware constraints. SR is an illposed problem as a low-resolution observation can correspond to multiple high-resolution counterparts. Deep learning methods address this by learning a non-linear mapping from a large dataset of low-resolution to high-resolution image pairs. In our work, we utilize the cardiac Magnetic Resonance Imaging (MRI) dataset from the Data Science Bowl Cardiac Challenge, which contains 2D cine images with a spatial resolution of 256x256 and $8-14$ slices per time-step, and approximately 30 volumes across a cardiac cycle for a single patient. We employ a Residual Dense Block-based 3D-CNN for spatial SR and a UNET-style Optical-Flow-based 3D-CNN for temporal SR. Initially, these models are trained separately, achieving Peak Signal to Noise Ratio(PSNR) and Structured Similarity Index(SSIM) of 25.06 and 0.9104, respectively, compared to a Baseline of 22.42 and 0.8223. Subsequently, we finetune these models jointly in a cascaded manner, which improves PSNR and SSIM to 27.53 and 0.9439, respectively. Further joint finetuning within a Generative Adversarial Network (GAN) framework, where the models serve as the Generator and a separate Discriminator distinguishes between real and generated images, results in PSNR and SSIM increasing to 27.63 and 0.9450, along with enhanced perceptual quality. Our results demonstrate that jointly finetuned deep learning models hold significant promise for advancing super resolution in medical imaging, potentially leading to improved diagnostic capabilities and patient outcomes.