Enhancing Traditional Kaldi Dysarthric Speech Recognition Using SSL-Features

Dysarthric speech poses significant challenges for automated recognition systems, which perform well with normal speech. This paper investigates the adaptation of embeddings from large self-supervised learning (SSL) models—Wav2Vec2, HuBERT, and Data2Vec—to traditional Kaldi-based DNN/HMM and Factorized-TDNN ASR systems for dysarthric speech. Baseline systems were developed using MFCC features, ensuring speaker and severity independence. Additionally, ASR inferences from the three pre-trained SSL models were used as a secondary baseline. Our study reveals that HuBERT features outperform the others on the TORGO database, while Wav2Vec2 excels in low-severity cases and HuBERT in medium to high-severity cases on the UAspeech database. We observed absolute improvements of 34% and 28.6% over zero-shot decoding results for the TORGO and UAspeech databases, respectively. These findings suggest that using SSL models as feature extractors is more effective than using them directly as ASR systems for dysarthric speech.