Hybrid-Based Relightable 3D-Aware Generative Model

Lighting-based 3D-aware generative networks have a multitude of applications in computer vision and graphics. However, the problem of generalization remains unresolved. In this paper, we propose a novel hybrid disentangled method for lighting control. We introduce a lighting control module into the pretrained model that disentangles illumination from geometry and appearance in the latent space. For the intermediate illumination components, a physically-based illumination model is employed to explicitly represent the lighting, thereby generating relit images through physically-based rendering. To reduce the computational cost associated with the physically-based illumination model, we use a neural bidirectional reflectance distribution function (BRDF). Our method enhances generalization capability without requiring additional supervision of intermediate components. We have experimented with the method on the FFHQ dataset and have outperformed the current method in terms of lighting accuracy.