Comparative analysis of different time-frequency image representations for the detection and severity classification of dysarthric speech using deep learning

Dysarthria, a speech disorder resulting from neurological damage, presents significant challenges in clinical diagnosis and assessment. Traditional methods of dysarthria detection are often time-consuming and require expert interpretation. This study analyzes various time-frequency image representations of TORGO dysarthric speech to facilitate the automatic detection and classification of dysarthria severity through deep convolutional neural networks (DCNN). The dysarthria detection problem was approached in experiment E1, a binary classification task involving dysarthria and a healthy control class. Experiment E2 employs a multiclass classification method, categorizing data into very low, low, medium, and healthy classes. The analysis of time-frequency image representations of speech features is presented in two forms: standard-form images, including cepstrogram and spectrogram, and compact-form images, such as cochleagram and mel-scalogram. The highest ranked feature is benchmarked with existing work for both dysarthria detection and its severity classification. And this proposed work analyzes the frequency behavior of time-frequency image representations by bifurcating the standard-form images into two halves: one half representing low frequency and the other half representing high frequency. By this approach, the bifurcated standard-form of cepstrogram with low frequency outperforms all other features by achieving a validation accuracy of 99.53% for E1 and 97.85% for E2 and this surpasses existing benchmark work by 6% for E1 and by 1.7% for E2 in the TORGO dataset. The best ranked feature of E1 and E2 was applied to the noise-reduced UASpeech dataset and the DCNN model achieved 98.75% accuracy in E1 and 95.98% in E2, demonstrating its effectiveness on new dataset. • The basic research question of this study is: will CNN learning of standard form image representations of dysarthria perform better than CNN learning of compact form image representation? • In this work, the MFCC-based CNN method is used as a benchmark to show that CNN learning of standard-form images outperforms MFCC-based CNN learning. • The time-frequency image representations consist of two forms: standard form image representations such as cepstrograms and spectrograms, and compact form image representations such as cochleagram and mel-scalogram image representations. • Two experiments were conducted: (E1) a binary classification problem to discriminate between healthy and dysarthric speech, and (E2) a multiclass classification problem to classify the severity level of dysarthric speech with the severity of very low, low, medium, and healthy classes. • To analyze the frequency behavior of dysarthria detection and its severity classification in the CNN, the standard-form images were bifurcated into low- frequency and high-frequency components. This leads to a third form known as bifurcated standard-form images, which comprise a bifurcated spectrogram and a bifurcated cepstrogram. This bifurcated standard form of the spectrogram and cepstrogram consists of low-frequency and high-frequency spectrograms, respectively. • All the image representation are applied for both E1 and E2 and thus totally 16 experiments are conducted and ranked based on the classification accuracy. • The result concludes that low-frequency of bifurcated standard form yield an accuracy of 99.53% for E1 and 97.85% for E2 and outperforms all methods including the existing state-of-the-art.