DEEP LEARNING-BASED MRI TO X-RAY SYNTHESIS OF KNEE JOINT FOR AUTOMATIC OSTEOARTHRITIS CLASSIFICATION

Every Imaging modality provides unique anatomical information and thus, multiple imaging of the same anatomy is often prescribed to understand and diagnose a disease. Due to restrictions in acquisitions, time, space, costs, availability, and risk factors like ionizing radiations, researchers sought to synthesize images from one modality to the other. Synthesis of CT from MRI and vice-versa has been performed using many Deep Learning-based models such as Autoencoders, U-Net, Generative Adversarial Network (GAN) architectures, and Transformers. However, synthesizing X-rays from MRI has not been reported. Knee osteoarthritis is typically diagnosed using X-rays while MRI is also typically acquired to understand the extent of cartilage degeneration. Synthesizing X-rays from MRI would eliminate the need for X-rays in such a scenario. Synthesizing CT images from MRI is a domain transfer problem in computer vision. GAN architectures such as supervised conditional GAN or cycle GAN have been proposed [1] to synthesize CT images from MRI images. Here we use Cycle-GAN and Multi-Cycle GAN to synthesize knee X-rays from MRI. Cycle GAN [2] architecture typically consists of two generators and two discriminators where two generators (G A and G B ) are U-Net++ architectures and two discriminators (D A and D B ) are classifiers (Figure 1). G A synthesizes fake X-rays from Real MRI, while G B synthesizes fake MRI from fake X-rays. Discriminators D A and D B enlarge the difference between the fake and real images. Cycle consistency loss is used to optimize the X-ray generation and is determined by comparing fake and real MRI. A Multi-Cycle GAN [3] has an additional generator over and above what we have in Cycle GAN. The purpose of this additional generator is to control the consistency of the generated image in the target domain, improving the generation effect of the target generator. We developed Cycle GAN and Multi-cycle GAN architectures for synthesizing knee X-ray from MRI. An already available paired imaging dataset from Osteoarthritis Initiative (OAI) repository was used for this purpose. As expected, Multi-Cycle GAN was more precise and retained high anatomical consistency than Cycle GAN.