A Text-to-Speech Synthesis Method Based on Speaker Transfer

In the context of limited sample sizes for speaker voice data, end-to-end Text-to-Speech (TTS) synthesis models often face challenges related to synthesis quality, training speed, and inference time. To address these challenges, this study proposes an advanced speaker-adaptive TTS synthesis framework. The approach involves pre-training in a high-resource single-speaker monolingual domain, followed by fine-tuning in a low-resource single-speaker monolingual target domain. By masking the high-resource single-speaker embedding layer in the model, we achieve speaker timbre adaptation for the low-resource speaker through transfer learning. To enhance prosody in the generated speech, we incorporate prosodic features extracted and integrated from text using RoBERTa. Additionally, to improve synthesis efficiency, we introduce inverse short-time Fourier transform into the upsampling part of the decoder, significantly accelerating the synthesis process. The proposed model achieves a real-time factor (RTF) of 0.021, a speaker embedding cosine similarity (SECS) of 0.80, and a mean opinion score (MOS) of 3.34, demonstrating advancements in both synthesis speed and quality.