Multi-Modal Image Classification for Parkinson Disease Detection

Parkinson disease (PD) is a typical condition which affects the daily life activities like movement, speech, tremor, and impaired balance. As the disease progresses, people may have difficulty walking, talking, and completing other tasks. PD occurs when nerve cells in the brain do not produce enough dopamine, a brain chemical. Unfortunately, there are no medications for this condition. It can be handled through lifestyle changes. Hence, the study of Parkinson’s disease is essential and early detection will improve quality of life. Here, a multimodal classification method is applied to handle symptoms represented in a csv file and brain conditions represented as MRI images. Features from csv file and MRI images are extracted and classified using Deep Neural Network (DNN) and ViT models, respectively. The results of the DNN and ViT models are aggregated for the classification of PD disease. The accuracy of each model are 95.61 and 96 respectively. The results of these ensemble models in terms of accuracy, F1 score and AUC curve are 95.84, 0.9180 and 0.9592 respectively. The proposed multimodal classifier can detect Parkinson disease with confidence above 90% considering various factors into account.