Enhancing Classification and Reasoning Abilities of Large Language Models for Predicting Affect from Ambulatory Data

This study investigates the potential of large language models (LLMs) to leverage their extensive pre-trained knowledge for conducting affect classification using ambulatory data collected from smartphone sensors in support of precise mental health (MH) monitoring. We propose several prompting strategies to conduct a three-way classification of happiness and sadness levels: baseline, temporal method, personalization method, and two variations of a combination method to guide the learning process. These designs incorporate in-context learning, prompt engineering, and additional MH-related context to enhance model performance. Results indicate that the combination strategy outperforms the others in terms of balanced accuracy and macro F1 score for both the sadness and happiness tasks using longitudinal GPS, phone log, and accelerometer data. Additionally, we conduct a linguistic analysis of the LLM’s reasoning that led to its decision across these strategies. The findings suggest that prompting methods where the LLM reasoning incorporates language related to affect, analytical thinking and authenticity yield improved affect classification performance.