Efficient Zero-Shot Voice Cloning for Bengali Speech Synthesis

Voice cloning has numerous useful applications, including assisting individuals who have lost their ability to speak, movie dubbing, and translating voices into different languages. However, voice cloning in Bangla is still in its early stages. In this work, we explore two standard approaches for zero-shot Bengali voice cloning, which can generate cloned audio from short samples, even for Bengali speakers not included in the training data. Despite using limited datasets, our models achieve cloning quality comparable to state-of-the-art models trained on much larger datasets, maximizing potential benefits for the Bengali community. We use a neural network technique called the speaker encoder (SE) voice cloning method, which achieved a Mean Opinion Score (MOS) of 3.8 and 83% cosine similarity between the original and cloned voice. This model was trained on a custom multi-speaker dataset. Additionally, we employ a speaker converter (SC) approach trained on a single-speaker dataset, which achieved a MOS of 4.0 and 82.64% cosine similarity. By combining the two models in an ensemble, we further improved performance, reaching an MOS of 4.4 and a cosine similarity of 85.21%. Our proposed approach can be an important step toward developing voice cloning technology for the Bengali language, potentially positively impacting the community. The demos can be accessed here.