Human activity recognition engineering

Physical activity is important to our overall health. The analysis of movement can lead to enhanced physical well-being by informing injury prevention and guiding rehabilitation. Human Activity Recognition uses sensors to detect and classify these movements automatically, a practice gaining traction in healthcare. However, there are several challenges that reduce the classification performance of activity recognition methods, such as the quality of recorded data, the shortage of large datasets, and the absence of standardized workflows. This thesis proposes targeted solutions to address these challenges, with each contribution focusing on a distinct aspect of the problem: (1) To enhance data quality and accurately distinguish between movements, our approach involves identifying optimal sensor placement. By converting camera images into two-dimensional human pose estimations, we identify potential sensor locations based on the derived pose data. The pose data are then input into a mathematical model, which produces a similarity score quantifying the distinction capabilities between movements based on the sensor's location. This methodology further facilitates the integration of pose estimation data with sensor data, supporting a multimodal classification approach. Evaluation across different datasets confirms that careful sensor positioning and data integration improves classification performance. (2) To overcome data scarcity and enhance data utility, this work introduces two novel data preparation techniques. First, sensor data is transformed into movement images suitable for computer vision applications. Second, synthetic data, generated using animations in a gaming engine, provides a cost-effective and scalable alternative to traditional data collection methods. Using benchmarks, we demonstrate that data augmentation and transfer learning boosts the classification outcome, especially with small datasets. (3) To standardize the workflow and foster user-engagement, a human-in-the-loop software framework using real-time feedback was developed. This framework informs users about the classification accuracy of their activities, allowing the recollection of data for specific activities as needed. Furthermore, a classification model can be trained on-device, enabling personalization. This process of user-driven data refinement and on-device model training adapts the system to individual patterns, as demonstrated on different datasets. In summary, this work proposes an integrated approach by unifying strategic sensor placement, advanced data augmentation techniques, and real-time feedback. The research contributes to advancing Human Activity Recognition technology, making it more accessible and applicable across various fields.