TTSDS2: Robust Objective Evaluation for Human-Quality Synthetic Speech

Recent near-parity TTS systems sound so natural that objective evaluation is more challenging than ever. This paper presents TTSDS2, a robust objective evaluation measure. TTSDS2 is an improved, fully unsupervised version of the Text-to-Speech Distribution Score that needs only a small pool of reference and noise data. We conduct what we believe is one of the largest listening test evalations of human-quality synthetic speech to date: we collect MOS, CMOS and SMOS judgements from 200 raters, scoring 20 recent voice-cloning models over four domains: clean and noisy audiobooks, in-the-wild YouTube speech, and children’s dialogue. We compare 16 TTS objective metics, finding that across all 12 domain–score pairs, TTSDS2 is the only objective metric that achieves a Spearman correlation above 0.50 with respect to every one of the subjective scores (mean 0.67). TTSDS2 therefore offers a robust, interpretable yardstick for future TTS research.