Just Breathe: Harnessing Pretrained Audio Models for Mood Awareness in a Gamified Breathing Training App

Mood disorders can significantly impair an individual’s ability to function in daily life. One common symptom observed in early stages of several mood disorders is Agitation. Early detection and regulation of agitation can thus help prevent the progression of severe mood disorders. Digital Health Interventions such as guided slow-paced breathing apps have demonstrated their efficacy in reducing agitation. However, such interventions are often hindered by poor adherence in the long term. We hypothesize that an awareness of the individual’s current mood states would allow for a more personalized intervention (e.g., adaptive session duration), thereby improving adherence. In this paper, we explore the feasibility of predicting real-time agitation in an individual as they engage with a gamified biofeedback-based breathing training app Breeze. In a controlled lab experiment (N = 30), we collect voice and breathing samples from Breeze alongside self-reports of perceived agitation. Two pretrained audio models, VGGish and OPERA, are used to extract feature embeddings from raw audio signals, and downstream classifiers are subsequently trained to predict agitation levels. Initial results show that (1) respiratory audio is indeed a reliable predictor of agitation, and (2) pretrained models are capable of extracting meaningful features from respiratory audio. Our findings lay the technical groundwork to further investigate and evaluate the effect of mood-aware personalization strategies to improve user adherence, and ultimately, lead to better mental health outcomes.