Speech Synthesis Method of Virtual Liver Based on Tacotron and Conditional ß- Variational Autoencoder

Virtual liver has become a emerging tendency in the fields of new-media and social network. In the process of building bright individuality image for virtual liver, how to generate more vivid speech is still a challenging problem, especially for stylization and diversity. In this paper, we take First Impression V1 dataset as training data. Meanwhile, multiple models including DeepFace, InsightFace, and FaceNet are utilized to achieve the mapping relations between liver's attributes and the speeches in videos. These attributes consist of facial appearance, emotion, body mass index, age, gender, and big-five personality traits. Afterwards, we propose a novel speech synthesis method of virtual liver by hybridizing multi-scale channel attention module and conditional p-variational autoencoder. Through application of the proposed model, we can obtain diverse synthesized speech with various settings of individual's attributes. Moreover, we also investigate in different attributes' influence on synthesized speech.