Fully End-to-End Chinese-Tibetan Bilingual Speech Synthesis

In this paper, we address the problem of insufficient amount of Tibetan speech data for bilingual speech synthesis. Based on the commonality of Chinese and Tibetan languages, we investigate a hybrid Chinese-Tibetan phoneme conversion table to share model training primitives. And then an end-to-end Chinese-Tibetan bilingual speech synthesis model is trained using the vits model. The model is capable of achieving cross-language speech synthesis for speakers utilizing merely two monolingual corpora; however, the Mean Opinion Scores (MOS) indicate a slight deterioration in the naturalness of the synthesized speech. In addition, this paper investigates the effects of whether Tibetan phonemes are labeled with tones or not, whether the bilingual speech synthesis model uses a hybrid phoneme list or not, and the training set size on the quality of speech synthesis. The experimental results show that for bilingual speech synthesis training, although adding tones to Tibetan can improve the performance of speech synthesis for a specific monolingual, it is difficult to fully utilize its performance because of dataset size; after that, this paper finds that mixed tones outperform other representations, and different language complexities lead to different amounts of data required for training.