Hierarchical Deep Learning for Human Activity Recognition Integrating Postural Transitions

Data scarcity in human activity recognition (HAR) datasets can often lead to overfitting on particular components of the data. This article implements stacked 1-D convolutional long short-term memory (LSTM) models to leverage the inherent hierarchical nature of the data by utilizing a similar hierarchical structure, combining multiple models for inference. This helps to overcome the issues of data scarcity that are inherent in these forms of data, in particular, postural transitions (PTs). PTs are a fundamental indicator of at-home independence but are often neglected from HAR datasets and studies. We train and compare our network performance on the raw data, without feature generation, of three open datasets that specifically contain this modality, which is often not included due to its scarcity. The hierarchical CNN-LSTM achieves accuracy in line with current state of the art, with the accuracies of 92%, 84%, and 94% on the UCI-HAPT, KU-HAR, and UniMiB. It also achieves a consistent F1 score of 0.90 and a Cohen’s Kappa of 0.90, highlighting the network’s ability to achieve agreeable and reliable results on a range of different datasets. The framework was validated with both k-fold and an 80:20 train–test split. The work also highlights that the small size and inference time make this network architecture a candidate for on-device deployment.