Few-Shot Human Activity Recognition Using Lightweight Language Models

The lack of labeled data and model generalization abilities have historically represented major obstacles for Human Activity Recognition (HAR). Few-Shot Learning (FSL) addresses both of these issues. However, its application to HAR, is particularly difficult, since transferring models into new environments requires the model to adapt not only to a new set of activity labels but also, in many cases, to a different sensor configuration and possibly even a different sensor modality. The ability of Large Language Models (LLMs) to understand and interpret natural language, together with their versatility as few-shot learners, can greatly simplify model transfer. This is particularly the case where sensor activations can be transformed into text descriptions, since such descriptions abstract from the specific sensor modality and configuration. Unfortunately, the application of LLMs at the edge is typically hindered by their hardware requirements, making it unfeasible or highly inefficient to deploy these models to be used locally within smart environments. In this context, we propose a lightweight LLM-based FSL approach to facilitate model transfer with only a few labeled data samples, while using a model small enough to be deployable at the edge. Our results show that a relatively small BERT-based LLM architecture (hundreds of millions of parameters vs. hundreds of billions of parameters) can outperform larger models (including Chat-GPT). The approach was evaluated on two challenging datasets, namely the “van Kasteren (VK)” and the “VK houses” datasets. Using our 10 -shot FSL approach, we obtain a macro average F -score of $52.75 \%$ on “VK” vs. a baseline F -score of 26.63% using Chat-GPT. On the “VK houses” dataset, our macro average $\mathbf{F}$-score is $\mathbf{3 6. 0 2 \%}$, in contrast to $\mathbf{1 5. 4 6 \%}$ for Chat-GPT.