Encoding Human Activity Recognition: Local Coordinate Coding on Multimodal Sensory Data

Human activity recognition (HAR) aims to recognize the human daily life actions from videos or sensory data. Recently, the rapid spread of smart devices, equipped with diverse sensors, has enabled the continuous recognition and monitoring of human activities. A critical challenge in HAR lies in the computation of features that accurately capture the meaningful characteristics of each activity sequence for precise recognition. Crafting such features through domain knowledge and in-depth sensor data analysis can be laborious and potentially error-prone. To address this limitation, we have proposed an HAR framework that uses the codebook-based approach for feature learning. The proposed method works in three steps: codebook construction, feature encoding and model training. In the first step, the raw sensory data is segmented into smaller subsequences. These subsequences are grouped into clusters by using MiniBatch k-means clustering. The center of each codebook is a codeword which represents a statistically distinct subsequence. In the next step, each subsequence is assigned to the nearest codewords by using Local Coordinate Coding (LCC), encoding each sequences as encoded vector that captures the distribution of these codewords within a sequence. Finally, these encoded feature vectors are fed into classifier for the activity recognition. The performance of the proposed method was evaluated on publicly available WISDM dataset using four metrics: accuracy, precision, recall and $\mathbf{F 1}$-score. The proposed method was able to achieve the accuracy of $\mathbf{9 7 \%}$. The results were compared with the existing state-of-the-art techniques on this dataset, which shows the effectiveness of our proposed method.