Text-Based Audio Correction Using Deep Learning Framework

This research paper presents an innovative approach to audio correction using a Deep Learning Framework that leverages Text-to-Speech (TTS) and Automatic Speech Recognition (ASR) models. The TTS Tortoise model synthesizes speech from conditioned text, employing advanced neural network techniques to generate realistic and high-fidelity audio output. The lightweight Whisper Tiny ASR model accurately transcribes user voice input into textual format, forming the initial step in the audio editing process. Content creators commonly face challenges in rectifying errors in audio recordings. This project presents a practical framework for text-based audio editing, enabling creators to effortlessly correct errors and refine audio content without resorting to time-consuming re-recordings by manipulating the accompanying text. While existing approaches have offered solutions to this problem, the proposed framework provides an end-to-end solution, incorporating practical implementation methods. The framework’s efficacy is demonstrated through studies reporting mean opinion scores (MOS) for the Tortoise TTS model in the range of 3.5–4 on a 5-point scale, indicating good to excellent voice quality. This paper outlines an effective approach to addressing audio editing challenges, emphasizing the practicality of the proposed framework in solving real-world problems for content creators.