Towards the Virtual Reconstruction of Physically Lost Altarpieces through Differentiable Rendering

The interest of human beings in artistic creation, and its preservation, has been a constant throughout history. Thanks to this, today we can enjoy works from various moments and origins. However, not all works have been so fortunate, and many have disappeared for various reasons. To mitigate this loss, the virtual recreation of lost works of art is an open topic of interest for the computer graphics and cultural heritage communities. In the literature, we can find works along this line that are based on manual modeling or the use of numerous modern photographs. In this interdisciplinary work, however, we present a new methodology for reconstructing destroyed altarpieces for which only a single historical image is preserved, leveraging modern techniques of differentiable rendering. Additionally, we test our methodology in the reconstruction of the altarpiece of Sant Joan Baptista (Valls, Spain) destroyed during the Spanish Civil War. Our approach results in the partial 3D reconstruction of the main reliefs of the altarpiece. These results are truly useful, both for experts and for the public, as they allow for a better understanding of its volumetrics and its relationship with space, representing an important step forward in the virtual recovery of lost artifacts.