3D Face Reconstruction from Sketches with Hybrid Model Architecture

Recent advances in 3D face reconstruction have shown impressive results. However, these methods predominantly focus on real face images, leaving a gap in accessible and user-friendly 3D face modeling options. To address this challenge, we introduce our hybrid model approach, an end-to-end 3D generation framework tailored for sketching, enabling precise face reconstruction from a single sketch. Using a hierarchical approach, our system predicts missing facial geometry to enhance the extraction of fine geometric features. The resultant predictions combined with the original inputs guide the learning process of the 3D face model. To mitigate prediction uncertainty, our model incorporates a novel “predict consistent” loss, refining reconstruction outcomes with increased detail while closely adhering to the input sketch. Rigorous experiments demonstrate the capability of our method to produce high-quality face models from sketches.