NeuRiPhy: Neural Baking of Physics-Based Deformations for Facial Rigs 59

We introduce a self-supervised neural approach for real-time, physically-principled facial rigs that incorporate soft tissue deformations and contact interactions. Traditional artist-crafted blendshapes lack the capacity to produce realistic tissue deformations, while physics-based models, despite their accuracy, are computationally prohibitive for interactive applications. Our method addresses these limitations by learning a neural map from a set of rig controls to corresponding deformations that minimize the mechanical energy of an anatomically-based face model, including soft tissue, skull, and teeth layers. This self-supervised framework eliminates the need for predefined simulation data and supports unsupervised handling of collisions between rigid and soft bodies, streamlining the computational process. We demonstrate that, for the first time, our approach achieves real-time performance of physics-based face rigs with complex non-linear deformations and contact handling.