Gradient Guided Depth Super-Resolution Using Attention-Based Cross Domain Interaction Module

Color guided depth image super-resolution (DISR), leveraging the strong structural similarity between the registered high-resolution (HR) color image and the low-resolution (LR) depth image. has achieved remarkable results. However, when the sampling factor is large, the DISR suffers from reconstructing accurate depth edges due to the severe loss of high frequency components. Crucially, direct use of the high frequency (HF) information of the registered map may transfer the high-frequency textures into the depth results and blur the depth edges. To address this, we propose GDISRNet, which is a gradient guided DISR network. It not only utilizes HF information of the color image, but also fundamentally avoids the texture copying artifacts. First, we compute gradient map from the Y channel of HR color image. Then, an Attention-based Multi-Scale Feature extraction module (AMSF) extracts the multi-scale features respectively from gradient map and the interpolated LR Depth map. Moreover, a novel Cross-Domain Interaction Module (CDIM) fuses these features to enhance depth boundaries while suppress color textures at the same time. Third, a gradient loss introduces to prevent edge smoothing effectively. Finally, a residual progressive structure of AMSF and CDIM reconstructs the SR depth image with enhanced high-frequency information. Experiments on several popular benchmark datasets including <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Middlebury</i>, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Sintel</i>, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Lu</i> and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">NYU V2</i>, demonstrate the superiority of the proposed GDISRNet over representative state-of-the-art methods.