AvatarMirror: Rendering Restoration for Real-Time Immersive Presence System

Human-Centered immersive computing is catching on within VR and AR. We establish a real-time immersive presence system called AvatarMirror that enables arbitrary characters to look at themselves instantly in novel stereo views. First, stereo matching is implemented to acquire multi-view depth priors, based on which the human topology is reconstructed through raycasting from a novel perspective. Then, the color information from the referenced views is adaptively blended to perform the human rendering. Furthermore, we propose an RR-Net to conduct rendering restoration. AvatarMirror realizes real-time high-fidelity immersive presence with multi-threaded scheduling and CUDA acceleration.