SmartTUG: deep learning-based automated TUG subphase segmentation and biomechanical analysis from monocular RGB video
The Timed Up and Go (TUG) test assesses mobility, balance, and fall risk, but routine practice relies on total completion time or visual judgment. Subphase segmentation could support more detailed movement assessment. This study developed and preliminarily evaluated an automated TUG analysis system using smartphone-recorded videos. The system integrates monocular pose estimation, deep learning–based subphase segmentation, and biomechanical measure extraction. The segmentation framework integrates spatiotemporal graph convolution and Transformer modules to capture local movement patterns and temporal dependencies, followed by task-constrained post-processing to refine phase predictions. The framework achieved 94.02% accuracy, 94.23% precision, 93.80% recall, and 93.53% F1 score. Friedman testing identified differences in accuracy among the compared deep learning models (p < 0.0001), with post hoc analysis showing the framework outperformed reproduced TUG segmentation models. Simulation experiments showed that post-processing improved accuracy under pauses and local speed changes, while performance remained close to baseline across tested capture conditions except substantial temporal subsampling. Model interpretation using SHapley Additive exPlanations showed that the body regions contributing to predictions were generally consistent with the visual events used for manual phase segmentation and t-SNE visualization showed clear phase-dependent clustering of learned features. Preliminary evaluation by rehabilitation-related professionals indicated generally positive perceptions of the system’s usefulness and ease of use. These findings highlight the promise of an explainable deep learning model for automatic TUG subphase segmentation from smartphone-recorded videos under standardized conditions. Validation in older adults and clinical populations is required before its use in clinical or daily-life settings can be established.
Authors
- Yanxin Zhang (ORCID: https://orcid.org/0000-0002-7638-1669)
- Wenqi Liang (ORCID: https://orcid.org/0000-0002-6814-9958)
- Rebecca Mary Meiring (ORCID: https://orcid.org/0000-0002-8472-4940)
- Hongbo Wang (ORCID: https://orcid.org/0000-0002-4805-3182)
- Xingye Cheng (ORCID: https://orcid.org/0009-0001-5246-6174)
- Junyi Cao
- Xi Gao
Institutions
- University of Auckland (NZ)
- Xi'an Jiaotong University (CN)
Publication Details
- Journal
- Biomedical Signal Processing and Control
- Published
- 2026-11-21
- DOI
- https://doi.org/10.1016/j.bspc.2026.111904
- Primary Topic
- Balance, Gait, and Falls Prevention
- Type
- article
- Field-Weighted Citation Impact
- 0.00