Voice-Based Assessment of Extrapyramidal Symptoms Using Deep Learning

Extrapyramidal symptoms encompass features of Parkinsonism, including bradykinesia, cogwheel rigidity, and resting tremors, which contribute to motor impairments hindering handwriting and speech. In this study, we analyzed voice data captured using a voice sensor setup from 94 patients exhibiting varying levels of EPS and 30 unaffected controls. Each participant provided 13 recordings of repeated vowel and consonant sounds. The Drug-Induced Extrapyramidal Side Effect Scale and Glasgow Antipsychotic Side Effect Scales were used when grading patients into mild, moderate, and severe extrapyramidal symptoms, both administered by trained clinicians. To develop an objective assessment tool, we employed a transfer learning approach using a DenseNet architecture for feature extraction and classification. Its architecture enables the hierarchical concatenation of features at each layer. In this study, we identified that key acoustic features, MFCC, chroma, and spectral contrast vary significantly with the severity of extrapyramidal symptoms. Based on these findings, we developed a DenseNet-based model capable of predicting extrapyramidal symptoms from voice data. This model can classify with an accuracy of 81.9% and a precision of 82.0%. To the best of our knowledge, this is the first study to introduce a voice-based model for assessing the severity of extrapyramidal symptoms.