Estimating Parkinson’s disease finger tapping severity from RGB videos using a temporal–statistical–spectral feature fusion framework

Parkinson’s disease (PD) is characterized by progressive motor impairment, and finger tapping is widely used in the MDS-UPDRS (Movement Disorder Society-sponsored revision of the Unified Parkinson’s Disease Rating Scale) to assess bradykinesia. However, clinical scoring is limited by inter-rater variability and brief observation windows that may not capture fluctuations in motor performance. Most existing automated approaches rely on summary statistics of speed and amplitude from a single thumb–index joint, leaving frequency-domain dynamics and multi-joint coordination largely unexploited. We analyzed 189 finger tapping videos from 97 patients with PD and propose a multi-branch encoder framework that jointly models temporal, statistical, and spectral characteristics of multi-joint hand kinematics from standard RGB videos. Angular velocities of 19 hand joints were extracted via a pose estimation model and processed by three parallel encoders: a bidirectional temporal convolutional neural network, a statistical encoder, and a three-dimensional convolutional network operating on per-joint spectrograms stacked along the joint dimension. The fused representation was used for three-class severity estimation under a nested cross-validation to ensure robust evaluation. The framework achieved an AUROC(Area Under the Receiver Operating Characteristic Curve) of 0.860 ± 0.024, improving AUROC by up to 0.096 over prior methods re-evaluated on the same dataset. Interpretability analyses demonstrated that both joint-level and frequency-domain features contributed to severity estimation, with distinct patterns observed across severity levels. These results indicate that integrating temporal, statistical, and spectral features across multi-joint hand kinematics enables more comprehensive video-based quantification of PD bradykinesia, supporting practical digital biomarker development from standard RGB recordings without specialized equipment.