A Case Study on the Construction of an AI Training Dataset for Natural Korean Speech Generation by 3D Digital Humans

Recently, the demand for 3D digital humans—capable of being used without spatial or temporal constraints—has been increasing in broadcasting and content production. However, there is a lack of 3D facial data tailored for natural Korean speech generation. This study aims to construct a Korean speech-driven 3D facial dataset for digital human creation and to develop a generative AI model that produces realistic facial animations. A total of over 570,000 high-resolution 3D meshes were built through multi-view capture of 5,000 carefully designed sentences that reflect articulatory features and phoneme distributions. Based on this dataset, a transformer-based CodeTalker model was trained to generate synchronized and lifelike 3D facial animations from speech input. This research establishes a foundation for Korean-specific digital human development and contributes to various domains such as the metaverse, broadcasting, and public services through the practical application of generative AI technologies.