Two-shot Spatially Varying Bidirectional Reflectance Distribution Function Estimation for Lustrous Material by Deep Learning

To realistically represent the appearance of real-world materials in computer graphics, it is essential to estimate the spatially-varying bi-directional reflectance distribution function (SVBRDF) of each material. Using deep learning, several convenient methods for SVBRDF estimation have been proposed. However, challenges remain in handling materials with highlights and patterns due to the simplicity of the model and limited input data. We focus on lustrous fabrics, a commonly seen material that exhibits highlights and patterns. To capture visual cues such as highlights and shading, we use a flash-lit image, and to obtain the base color of the material, we use a non-flash-lit image. We propose a deep neural network that takes these two images as inputs and estimates 13 types of SVBRDF parameters based on the Disney principled BRDF model. In the network's loss function, we use a rendering loss to compare rendered images of the predicted and ground-truth maps under multiple lighting conditions from various viewing directions. Additionally, we introduce a sheen mix rendering loss as an auxiliary loss function. For the training dataset, we use approximately 250,000 images, including diverse materials created by artists. Our method demonstrates quantitatively and qualitatively superior results across a wide range of materials, including lustrous fabrics that are challenging to capture with existing SVBRDF estimation methods.