DTR: A Unified Deep Tensor Representation Framework for Multimedia Data Recovery

Recently, the transform-based tensor representation has attracted increasing attention in multimedia data (e.g., images and videos) recovery problems, which consists of two indispensable components, i.e., the transform and the characterization. Previously, the development of transform-based tensor representation has focused mainly on the transform perspective. Although several attempts have considered shallow matrix factorization (e.g., singular value decomposition and nonnegative matrix factorization) for characterizing the frontal slices of the transformed tensor (termed the latent tensor), the faithful characterization perspective has been underexplored. To address this issue, we propose a unified <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">D</b>eep <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">T</b>ensor <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">R</b>epresentation (DTR) framework by synergistically combining the deep latent generative module and the deep transform module. Especially, the deep latent generative module can faithfully generate the latent tensor as compared with shallow matrix factorization. The new DTR framework not only allows us to better understand the classical shallow representations but also leads us to explore new representations. To examine the representation capability of the proposed DTR, we consider the representative multidimensional data recovery task and suggest an unsupervised DTR-based multidimensional data recovery model. Extensive experiments demonstrate that DTR achieves superior performance compared to the state-of-the-art methods from both quantitative and qualitative aspects, especially for fine detail recovery.