Improving reliability of movement assessment in Parkinson's disease using computer vision-based automated severity estimation

BackgroundClinical assessments of motor symptoms rely on observations and subjective judgments against standardized scales, leading to variability due to confounders. Improving inter-rater agreement is essential for effective disease management.ObjectiveWe developed an objective rating system for Parkinson's disease (PD) that integrates computer vision (CV) and machine learning to correct potential discrepancies among raters while providing the basis for model performance to gain professional acceptance.MethodsA prospective PD cohort (n = 128) were recruited from multi-centers. Motor examination videos were recorded using an android tablet with CV-based software following the MDS-UPDRS Part-III instructions. Videos included facial, upper- and lower-limb movements, arising from a chair, standing, and walking. Fifteen certified clinicians were recruited from multi-centers. For each video, five clinicians were randomly selected to independently rate the severity of motor symptoms, validate the videos and movement variables (MovVars). Machine learning algorithms were applied for automated rating and feature importance analysis. Inter-rater agreement among human raters and the agreement between artificial intelligence (AI)-generated ratings and expert consensus were calculated.ResultsFor all validated videos (n = 1024), AI-based ratings showed an average absolute accuracy of 69.63% and an average acceptable accuracy of 98.78% against the clinician consensus. The mean absolute error between the AI-based scores and clinician consensus was 0.32, outperforming the inter-rater variability (0.65), potentially due to the combined utilization of diverse MovVars.ConclusionsThe algorithm enabled accurate video-based evaluation of mild motor symptom severity. AI-assisted assessment improved the inter-rater agreement, demonstrating the practical value of CV-based tools in screening, diagnosing, and treating movement disorders.