FastSpeech2 Based Japanese Emotional Speech Synthesis

Modeling emotions is an important part of text-to-speech (TTS) research since its goal is to develop a technology for synthesizing naturally sounding speech. In this study, we aimed to build an end-to-end TTS system for the Japanese language that can synthesize emotional speech taking the FastSpeech2 as a base model. Most of the existing approaches use some form of conditioning the generation process on the emotion class label through a corresponding embedding. What separates such methods is the way this conditioning is implemented. Our approach is to add two new blocks which are essentially transformer multi-head self-attention blocks having an input combined with emotion embedding. They are inserted before and after FastSpeech2's variance adaptor and share parameters to ensure robust emotion conditioning when the amount of training data is small. The proposed model was objectively and subjectively evaluated using Mel-Cepstrum Distortion (MCD) and Mean Opinion Score (MOS) criteria respectively. The obtained results show that our approach performs better than a collection of emotion-specific models obtained by fine-tuning the base FastSpeech2.