GAINS : Syntes av mänsklig gaussisk animation från talinmatning

Generating talking human models from speech input has a long research history, involving advancements in both motion synthesis and audio-visual alignment. Meanwhile, 3D Gaussian Splatting has recently emerged as a powerful rendering technique for modeling complex scenes with high visual fidelity. With more and more research attention, applying 3D Gaussian Splatting to human modeling has now become increasingly feasible in real-time scenarios. This thesis proposes GAINS : an end-to-end method that directly generates animated 3D human models from natural speech input, using 3D Gaussian Splatting as representation method. Unlike traditional multi-stage pipelines that typically rely on third-party tools for video processing, motion capture, or manual annotation, the proposed approach predicts SMPL-based motion directly from speech and synthesizes the corresponding 3D human motions. This significantly reduces system complexity while maintaining high rendering quality and motion fidelity. Through comprehensive evaluations using PSNR, SSIM, and LPIPS metrics against recent methods that rely on rendering frameworks like Pytorch3D,Pyrender,Blender and Nerf, experiments demonstrate that GAINS produces higher-quality animations with superior visual fidelity and greater perceived realism compared to recent multi-stage pipelines User study evaluations with 73 participants also confirm that GAINS human motions scored consistently higher for perceived realism and visual quality compared to conventional multi-stage generation pipelines. This work is among the first to directly generate speech-driven talking human motion using 3D Gaussian point representations, achieving real-time rendering performance.