Synergy of Eye and Hand: Multimodal Biomarkers for Cognitive Assessment
Traditional handwriting analysis for neurological assessment captures motor output but largely misses the guiding cognitive processes like visuospatial planning and attention. This study introduces a multimodal approach, combining online handwriting kinematics with concurrent eye-tracking data from 48 older adults performing the Pentagon Copy Test (PCT). We extracted novel feature sets, including Hand-Eye Coupling (HEC) and Fractional Derivative (FD) biomarkers, and used an XGBoost classifier with Recursive Feature Elimination (RFE) to predict a binarized PCT performance score. Our final model, integrating all features, achieved a Balanced Accuracy (BACC) of 90%, significantly outperforming a model trained on baseline features alone ($79 \%$ BACC). The findings demonstrate that integrating eye-tracking data with advanced handwriting analysis provides a powerful and holistic tool for objectively assessing cognitive-motor performance, highlighting its potential as a sensitive digital biomarker.
Authors
- Ján Mucha (ORCID: https://orcid.org/0000-0001-5126-440X)
- Jiří Mekyska (ORCID: https://orcid.org/0000-0002-6195-193X)
- Michal Gavenciak
- Marcos Faúndez-Zanuy (ORCID: https://orcid.org/0000-0003-0605-1282)
- Pau Ferrer-Ramos (ORCID: https://orcid.org/0000-0002-1163-2113)
Institutions
- Universitat Pompeu Fabra (ES)
- Brno University of Technology (CZ)
Publication Details
- Published
- 2025-11-03
- DOI
- https://doi.org/10.1109/icumt67815.2025.11268615
- Primary Topic
- Writing and Handwriting Education
- Type
- article
- Field-Weighted Citation Impact
- 0.00