Tracking Ephemeral and Residual Emotions via Earable Inertial Sensing During Media Consumption

The extended screen time with the exponential rise of mobile applications has raised concerns about its impact on users’ mental health and frequent episodes of sudden emotional outbursts. Conventional emotion detection systems rely on intrusive physiological data or computationally intensive visual processing devices leading to a usability barrier. To address this challenge, we propose a novel method to detect ephemeral (immediate) and residual (lingering) emotional effects induced by media content by observing facial expression changes during content consumption. Our approach is based on predicting various facial expressions across different emotional behaviors using an earable-based Inertial Measurement Unit (IMU) to detect the ephemeral state and then map it to the circumplex model of affect to predict the residual state. A pilot study validates the potential of the proposed earable-based solution to offer a non-intrusive, efficient alternative for identifying the state of emotional turbulence of the user during screen time interaction.