ClearAI: AI-Driven Speech Enhancement for Hypophonic Speech

Hypophonia is a common speech symptom related to Parkinson's disease, affecting human comprehension and effective communication. Unlike dysarthric speech, hypophonic speech is characterized by its low volume and breathy voice, which makes it challenging to be heard and understood by human and voice-controllable systems especially in noisy environments. Conventional speech enhancement techniques, primarily focusing on amplifying audio power or cancelling environmental noise, fall short in improving the intelligibility and perception for hypophonic speech. To enhance hypophonic speech, we present ClearAI, an innovative AI-powered technology to improve speech quality for individuals suffering from hypophonia. ClearAI first leverages voice conversion technology to create a parallel dataset composed of normal and corresponding hypophonic speech samples. Then, ClearAI incorporates a predictive model trained on augmented parallel data to estimate the optimal audio style from hypophonic speech to strengthen the audio intensity and enhance the speech patterns. Next, a speech restoration model is built on the generated parallel speech data to reconstruct clear speech from the style transferred speech. Our experimental results reveal that ClearAI leads to substantial improvements in audio intensity in both digital formats and over-the-air transmission. In addition, ClearAI successfully reduces the hypophonic speech recognition error rate by more than 30% in noisy environments. Our human test results also validate ClearAI enhanced speech has the best human perceptual quality compared with other baseline methods.