Real-Time Character Animation Generation via Deep Learning-Based Facial Expression Recognition Using Unity Sentis

This paper designs and implements an expression-driven real-time character animation generation system based on the deep learning tool Unity Sentis. The system integrates the expression recognition model and Unity real-time rendering, and realizes the function of real-time expression capture and application to 3D models. Traditional motion capture technology is generally based on rule-based animation mapping. In contrast, our research adopts a deep learning approach and performs a lot of optimization for mobile terminals. It achieves higher recognition accuracy, lower latency and higher frame rate on performance-constrained mobile terminals. We tested the robustness of the entire system, and the system has good stability in complex lighting environments and multi-role scenes.