Facial Image-to-3D Avatar Conversion: A Deep Learning Approach for Customizable Game Characters

Character customization systems are an integral part of role-playing games. The demand for highly customizable and expressive 3D characters is on the rise, particularly in the context of the increasing popularity of artificial intelligence agents and avatar technology. In this study, we propose a method to automatically generate 3D player game characters based on input face images. Specifically, we leverage Character Creator 4 models and parameterize their blendshapes to achieve realistic facial expressions. Facial features are extracted from the input single frontal character image through training an encoder, and converted into applicable blendshape parameters (both shape and expression). At the same time, we utilize a texture generation network to generate the required texture maps for the character head model from the input image. To bridge the gap between the input image and the head model, we selected key annotation points of the head model in Unity. In order to enable gradient computation and backpropagation during the rendering phase, we employ the PyTorch3D renderer and supervise the network with metrics such as shape loss and luminosity loss. This method not only provides users with an efficient and faster means of 3D character generation, but also the generated results are fully compatible with the rendering process in the game engine, allowing users to further edit the shape or expression of the character in Unity.