Neural modeling and re-ageing of bio-physical facial appearance

Realistic facial appearance modeling and editing have gained significant attention in computer graphics. Recently, several diffusion-based biophysical skin models have been developed to accurately simulate skin appearance by incorporating multiple spectral parameters that represent chromophores in human skin. These chromophores, which are key absorbers in both visible and invisible spectra, play a crucial role in revealing various facial features under different lighting conditions. In this thesis, we propose a hyperspectral BSSRDF skin model spanning a wide spectral range from 300nm to 1000nm, as well as a monitor-based practical measurement method. For parameter estimation, we introduce a three-stage LUT search to speed up the conventional LUT search. However, this approach is constrained by the discreteness of Lookup tables. To overcome this limitation, we present an Encoder-Decoder network. This network is further simplified to accept a single image under D65 illumination and output the corresponding skin appearance directly. This simplified network is particularly useful for skin appearance editing, where adjusting melanin and hemoglobin concentrations allows for simulating various skin tones, such as tanned, lightened, and flushed. Building upon skin editing, we simulate the age-appropriate skin tone changes. However, skin tone variation alone is not the most obvious feature in facial re-ageing. On the other hand, existing re-ageing methods often generate high-quality transformations while neglecting the skin properties changes. To address this issue, we propose a re-ageing framework that integrates approaches, enabling more realistic age simulations across a wide range from 10 to 80 years old. Additionally, we uplift this method to a 3D by incorporating a geometric network that reconstructs coarse facial shapes with head size control, followed by a lightweight refinement network to generate fine details. This work is the first in computer graphics to combine a biophysical skin model with re-aging, producing high-quality transformations in both facial appearance and geometry.