Spatial Priors and Uncertainty for Enhanced Reconstruction in Computer Graphics and Vision

Many tasks in computer vision aim to reconstruct unknown quantities from observations of a scene, such as estimating depth from stereo vision, generating intermediate video frames, or revealing hidden shapes from transient measurements. In recent years, deep learning and differentiable rendering have become the methods of choice to tackle problems in those domains, but both exhibit a need for extensive computational resources. This thesis introduces innovative strategies that explicitly integrate our understanding of specific problems into reconstruction algorithms via spatial priors and uncertainty representations, improving both their efficiency and output quality by taking advantage of domain-specific peculiarities. <br> We begin by detailing how integrating rough shapes as priors into stereo matching can refine the depth estimation. In general, for a scene with no prior information, a comprehensive search across all possible values is performed to regress the disparity map between both images. However, in certain scenarios, such as stereo rigs embedded in setups containing other cameras, additional information can be used to improve the reconstruction. Our approach employs an efficient computation of the visual hull to reduce the search range of stereo matching, which, combined with various optimizations tailored to this use case, enables the accurate computation of depth at high resolutions. <br> Furthermore, we explore frame interpolation of rendered sequences where - in contrast to established methods - it is possible to generate and use additional data from the intermediate frame if necessary. By predicting the uncertainty of the interpolation output and incorporating partial renderings as priors, we devise a novel two-step model based on the transformer architecture that enhances the quality of the interpolated frames even for challenging content, as demonstrated quantitatively and qualitatively through a user study. This approach facilitates replacing the computationally costly rendering of a full sequence with a cheap interpolation of partial renderings. <br> Lastly, we tackle Non-Line-of-Sight reconstruction and demonstrate how the efficient implementation of a backward pass of a model-driven approach can lead to accurate reconstructions while reducing the runtime from hours or days needed by the baseline approach to minutes. This enabled us to explore different priors, and we show results using Gaussian blobs with an optional color component and total variation regularized depth maps. To address scenarios where the model assumptions deviate too much from the circumstances of real-world measurements, we introduce a background network inspired by neural representations and showcase its utility in capturing the remaining uncertainties.