Movieformer: Motion-resolved 4D MRI reconstruction using a network with spatiotemporal attention

Motivation: Investigate utility of self-attention deep learning to exploit global temporal information in motion-resolved 4D MR imaging. Goal(s): Design a novel hybrid convolutional-attention network to reconstruct motion-resolved 4D images without explicit k-space data consistency. Approach: A hybrid Unet-style 4D reconstruction network was developed to incorporate windowed multiscale spatiotemporal multihead self-attention. Training and testing were performed on free-breathing data acquired on patients with abdominal tumors. Results: Spatiotemporal attention successfully captured motion in multiple dimensions with improved image quality relative to state-of-the-art XD-GRASP reconstruction. Impact: Self-attention deep learning mechanism can combine long-range spatial learning and global temporal learning to augment capabilities of convolutional networks for improved motion-resolved 4D MRI of mobile tumors.