Can LLMs Learn the Language of Inertial Sensors in Human Activity Recognition?

Human activity recognition using inertial sensors has gained significant attention due to its applications in healthcare, fitness tracking, and smart environments. The classification of human activities from data collected by inertial sensors, such as accelerometers and gyroscopes, relies on the identification of motion patterns embedded in multivariate signals consisting of a sequence of ordered and temporally correlated samples. The classification accuracy reached by state-of-the-art models suggests that inertial signals retain the key features needed to distinguish among different human activities. This, coupled with the remarkable ability of large language models to acquire language capabilities in a semi-supervised manner, leads to the question of whether these models can be trained on time series taken from inertial sensors used in human activity recognition, to learn the ’’language" of the sensors and eventually solve time series imputation tasks.In this study, the Bidirectional Encoder Representations from Transformers (BERT) model is trained on accelerometer data, transcoded as structured sentences within a tokenized codebook that preserves temporal and spatial relationships. The trained model is then used to reconstruct missing data chunks in accordance with the masked-language modeling paradigm. Experimental results show that the proposed approach outperforms traditional statistical approaches for time series imputation, suggesting that the model is capable of acquiring the domain-specific behavior of inertial sensors.