Consistent 3D Medical Image Inpainting via Implicit Representation and Multi-View Sampling

Medical image inpainting plays a crucial role in various clinical applications, such as metal artifact removal, surgical planning, and anomaly segmentation. However, existing methods often operate on 2D image slices when inpainting 3D images, leading to inter-slice discontinuity and an inability to fully utilize the 3D structural information. To address these issues, we propose a novel 3D inpainting network leveraging Implicit Representations and Multi-View Sampling (IRMVS-3D). IRMVS-3D harnesses the comprehensive 3D information by multi-scale Masked AutoEncoder(MAE) guided encoding, and aggregating global and local features via Fast-Fourier-Convolution(FFC) blocks. To tackle the inter-slice discontinuity problem, IRMVS-3D treats the entire 3D image as a continuous implicit function, predicting voxel intensities through implicit representation. Additionally, to improve training efficiency while ensuring the continuity, we introduce a multi-view sampling strategy. During inference, intensities of all voxels are gathered to reconstruct the complete 3D volume. Experimental results demonstrate that our method achieves state-of-the-art performances in inpainted image quality on each view, while effectively addressing the inter-slice discontinuity problem.