Essential depth map processing algorithms for real-time applications: an overview

RGB-D cameras provide both depth (D) and colour (RGB) data as the output simultaneously in real-time. The depth data provided by the camera typically contains imperfections, such as holes and noise. Improving the quality of depth data is a widely researched topic of practical importance for multimedia, computer vision, and robotics. Typically, RGB-D cameras provide color data with higher resolution and quality compared to depth data. In this paper, we propose to enhance the depth map, which can be a pre-processing step of real-time volumetric streaming. First, we survey and categorize the depth improvement methods into two, and then test them to improve the quality of the depth data in real-time. We present the empirical results, which are informative for selecting a suitable depth map refinement method. Our experiments show that there is a trade-off between improvement level and real-time capability of the methods.