Vrefine: A Self-Refinement Approach for Enhanced Clarity and Quality in Text-to-Speech Models

Driven by advancements in the Large Language Model (LLM), there has been significant global attention on integrating the Generative Pre-trained Transformer (GPT) concept into Text-to-Speech (TTS) technologies. However, existing TTS models face issues such as inconsistent training data quality and reliance on autoregressive models, which makes controlling the quality of audio generation challenging. To address these challenges, this study introduces a novel TTS framework known as TTS-Vrefine, which is a new type of TTS architecture based on a self-feedback mechanism, aimed at enhancing the inference capabilities of the model and the quality of generated audio. The Vrefine framework iteratively refines the output, allowing the TTS system to self-train using its own generated data, significantly improving the clarity and quality of the audio. It expands the training dataset and enhances the self-optimization potential of the audio generation model, reducing the Word Error Rate (WER) of the base model by 2.4%, increasing the Perceptual Evaluation of Speech Quality (PESQ) by 2.22%, and improving the Short-Time Objective Intelligibility (STOI) by 4.55%. Additionally, the architecture optimizes the use of low-quality resources through a self-refinement mechanism, effectively expanding the training dataset.