A Multimodal Dataset for Assessing Emotion, Stress, and Emotional Workload in Interpersonal Work Scenario

We present EmoWork, a multimodal, multi-label dataset designed to support emotion and stress detection in realistic interpersonal work settings. Interpersonal work-common in occupations such as customer service-often requires workers to regulate their emotional expressions in response to strong affective stimuli. These demands, shaped by organizational display rules, present a unique challenge for affective computing systems, particularly in scenarios where internal emotional states diverge from observable behaviors. Despite this, no public datasets exist that capture such dynamics of affect in naturalistic settings. To address this gap, we collected physiological, behavioral, and self-reported data from call center workers who engaged in role-play scenarios simulating customer service interactions with professional actors portraying dissatisfied customers. The dataset includes self-reported affective ratings, which are used as labels for classification, synchronized recordings from three wearable devices (i.e., Polar H10, Empatica E4, and Muse S), and features extracted from video and audio data. The EmoWork dataset advances affective computing by offering context-rich, multimodal data grounded in realistic interpersonal work scenarios.