Build LLM-Based Zero-Shot Streaming TTS System with Cosyvoice

LLM-based text-to-speech(TTS) system has becoming the new trend and SOTA due to its high naturalness and zero-shot capability. However, it relies heavily on training data, usually requires at least thousands hours of labeled audio. In this report, we describe how to use pretrained CosyVoice model, to develop a streaming TTS system which supports Indian English and Indian languages. Though the pretrained CosyVoice model has never seen such data, it shows good performance in both specific speaker TTS and zero-shot voice clone after finetuning with merely 280 hours data. Experiment on LIMMITS25 challenge shows that our system achieves 4.46/4.19/4.55 naturalness, and 4.29/4.34/4.27 similarity in track1/track2/track3 respectively, which ranked 1st in all tracks.

Authors

Publication Details

Published
2025-03-12
DOI
https://doi.org/10.1109/icassp49660.2025.10890358
Primary Topic
Brain Tumor Detection and Classification
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Build LLM-Based Zero-Shot Streaming TTS System with Cosyvoice

Zhihao Du, Huadai Liu, Yuxuan Wang, Hao Wang et al.
Brain Tumor Detection and Classification
article

Build LLM-Based Zero-Shot Streaming TTS System with Cosyvoice

Zhihao Du, Huadai Liu, Yuxuan Wang, Hao Wang, Tianyu Zhao, X. R. Lyu
article en

Abstract

LLM-based text-to-speech(TTS) system has becoming the new trend and SOTA due to its high naturalness and zero-shot capability. However, it relies heavily on training data, usually requires at least thousands hours of labeled audio. In this report, we describe how to use pretrained CosyVoice model, to develop a streaming TTS system which supports Indian English and Indian languages. Though the pretrained CosyVoice model has never seen such data, it shows good performance in both specific speaker TTS and zero-shot voice clone after finetuning with merely 280 hours data. Experiment on LIMMITS25 challenge shows that our system achieves 4.46/4.19/4.55 naturalness, and 4.29/4.34/4.27 similarity in track1/track2/track3 respectively, which ranked 1st in all tracks.

Affordable and clean energy
Openalex Percentile: Top 94%
Brain Tumor Detection and Classification
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Build LLM-Based Zero-Shot Streaming TTS System with Cosyvoice — Zhihao Du, Huadai Liu, et al. · (2025) | TGRS Research Map | TGRS