Constant Q Transform for Audio-Visual Dysarthria Severity Assessment

This paper describes an automatic dysarthria severity assessment system submitted for the MDSA2024 challenge. The system utilizes audio and visual features to classify the severity of dysarthria. We investigate the effectiveness of different acoustic feature combinations, such as Mel spectrogram, filterbank and constant Q transform. Different encoders for audio data processing are compared in the experiments. The results demonstrate that the Conformer encoder achieves superior performance compared to the Densenet encoder on pure audio features. We further explore model-level fusion by combining audio and visual embeddings, leading to improvements in precision’ recall, and accuracy metrics. The best-achieved score on the test set is 7.5579, with an individual F1-score of 0.7028. The findings suggest that the proposed system with Conformer encoder and model-level fusion holds promise for automatic dysarthria assessment.