Long-Range and Non-Stationary Encoding for Dysarthric Speech Data Augmentation

Data augmentation methods have been employed to address the deficiencies in dysarthric speech datasets, achieving state-of-the-art (SOTA) results in the Dysarthric Speech Recognition (DSR) task. Current research on Dysarthric Speech Synthesis (DSS), however, fails to focus on the special encoding of pathological features in dysarthric speech. The dysarthric speech is characterized by its discontinuous pronunciation, uncontrolled volume, slow speech, improper pauses, and excessive nasal sounds. Moreover, compared with typical speech, the dysarthric speech contains more non-stationary components generated by the explosive pronunciation, hoarseness, and air-flow noise during the pronunciation. In this paper, we propose a DSS model named the Long-range and Non-stationary Variational Autoencoder (LNVAE). The LNVAE estimates the unique acoustic parameters of dysarthric speech by encoding the long-range dependency duration of phonemes in frame-level representations of dysarthric speech. Moreover, the LNVAE employs the Gaussian noise perturbation within the latent variables to capture the nonstationary fluctuations in dysarthric speech. The experiments were conducted on the speech synthesis and recognition tasks using the CDSD Chinese and UASpeech English corpora. The dysarthric speech synthesized by the LNVAE achieved the best performance across 29 and 28 objective metrics in the Chinese and English datasets, respectively. The synthesized speech also received the highest score from speech rehabilitation experts in the MOS experiments. The Whisper model fine-tuned on the synthesized data, achieved the lowest CER on the Chinese CDSD dataset. Moreover, for the UASpeech dataset, we increased the data by only 0.5 times to fine-tune the DSR model, yet surpassed the current SOTA method, which uses four times more augmentation data, by 4.52%. https://zdpbuilder.github.io/LNVAE2025