Exploiting voice conversion in creating new TTS voices

The development of high-quality Serbian text-to-speech (TTS) systems for new speakers traditionally requires extensive phonetic and prosodic annotations, a process that is both time-consuming and resource-intensive. This paper explores a novel approach that leverages voice conversion (RVC) techniques to generate synthetic speech in the voice of a target speaker. In this scenario phonetically and prosodically annotated transcriptions of the source speaker could also be re-used for target speaker, i.e. RVC synthesized speech, as well. Four models were evaluated: two adapted with natural speech (30 and 3 minutes, respectively), and two adapted with 2.5h of RVC-generated speech based where RVC conversion models are trained using also 30 and 3 minutes of target speakers’ speech. Listening tests assessed speech naturalness and vocal similarity. Results indicate that RVC-generated data enables effective adaptation of multispeaker TTS models, outperforming direct adaptation with very limited natural data. Moreover, the amount of target speaker data used to train the RVC model had minimal impact on final synthesis quality. These findings highlight the potential of using RVC for low-resource speaker adaptation in TTS systems for Serbian.