Emotional Speech Synthesis via Transformer Reference Encoder and Emotion Intensity Control

In recent years, emotional speech synthesis has been widely used in human-computer interaction, virtual assistants, voice navigation and other fields. However, many of the generated effects of speech synthesis are not realized in a very natural, realistic and controllable way. For this reason, we propose an improved FastSpeech2, which combines a transformer reference encoder, an emotion classifier with a contrast learning mechanism, and introduces a learnable emotion intensity scalar to achieve a fine-grained control of the degree of emotion expression. Specifically, we adopt transformers to replace the traditional reference encoder structure to extract more globally aware emotion embedding from the reference speech. Meanwhile, the emotion classifier is introduced for classification supervision to guide the embedding to learn the emotion differentiation ability and incorporate the contrast loss. To enhance the controllability of emotion expression, we design a learnable emotion intensity control module, which enables the model to dynamically regulate the degree of emotion expression during the generation process. Experiments are conducted on an Emotional Speech Dataset (ESD) multi-speaker multi-emotion corpus, and the results show that the proposed method outperforms the comparison baseline system.