A deep‐learning‐based pipeline for automatic fusion of CT coronary angiogram and stress perfusion CMR

BACKGROUND: Accurate evaluation of coronary artery constriction and myocardial ischemia is essential for diagnosing and managing coronary artery disease (CAD). Combining CT coronary angiography (CTCA) and stress cardiovascular magnetic resonance (CMR) imaging allows examination of both coronary artery narrowing and myocardial perfusion. PURPOSE: To develop a deep learning pipeline that integrates CTCA and CMR images, which could help improve accuracy in identifying affected vessels and their associated myocardial territories. METHODS: The proposed pipeline included two deep learning models: one for automatic reorientation of 3D CTCA and another for left ventricle (LV) wall registration between CTCA and CMR images. A 3D spatial co-registration model, the reorientation spatial transformer network (Reorientation STN), predicted reorientation parameters for input CTCA volumes using ResNet18 and STN. A 2D nonrigid spatial deformation network (Nonrigid SDN) was trained for LV wall registration. Cross-modal supervision was employed during training. Evaluation criteria included aspect ratio (AR), Dice similarity coefficient (DSC), and long-axis deviation angles. The process involved quantifying LV wall perfusion on registered CMR images and extracting coronary arteries from reoriented CTCA images to fuse these results. The pipeline was trained and validated on 447 pairs of CTCA and CMR images from 75 patients and tested on 18 subjects. RESULTS: The pipeline achieved an AR of 0.94 ± 0.03, long-axis deviation angles of 1.19 ± 0.83 (axial) and 1.54 ± 0.79 (coronal), a DSC of 0.66 ± 0.04 for LV wall reorientation, and a DSC of 0.92 ± 0.03 for LV wall registration between CTCA and CMR. CONCLUSIONS: This automated framework successfully fuses cardiac CTCA and CMR imaging, demonstrating its potential effectiveness.