Cost-Efficient Fall Risk Assessment With Attention Augmented Vision Machine Learning on Sit-to-Stand Test Videos

Falls among the elderly and other at-risk populations pose a significant public health challenge, necessitating innovative methods to detect and intervene early. While automated fall risk assessment methods using wearable sensors or depth cameras have been proposed, the physical and psychological burden of wearing extra sensors, and the high cost of specialized equipment like Kinect, limits their practical adoption. To tackle this challenge, this paper presents a novel machine learning-based fall risk assessment approach called <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">FRAVM</small>, which operates on Five times Sit-To-Stand (FSTS) test videos captured with standard, widely available cameras to identify individuals requiring fall prevention interventions. To enhance the practicality of <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">FRAVM</small>, 3D pose estimation is applied to generate vision-rich 3D body keypoints, mitigating the challenges posed by restricted camera angles. Median-average filtering is used to reduce noise caused by video shaking and pose estimation inaccuracies, while a new Dynamic Time Warping (DTW)-based matching algorithm is designed to handle interference from irrelevant individuals appearing in the video. Furthermore, a novel Attention-augmented Spatial-Temporal Graph Convolutional Network (AST-GCN) is developed for reliably identifying the action in each frame, enabling accurate computation of key kinematic features for fall risk prediction. Experimental evaluations on a dataset of 450 FSTS test videos demonstrate the high performance of <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">FRAVM</small>, with detection accuracy, F1 score, and ROC-AUC achieving 88.64%, 88.58%, and 98.86%, respectively. The ablation analysis confirms that the components in <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">FRAVM</small> are essential for achieving optimal results.