Wearable Device for Real-time Assessment of Facing Time in Small Group Social Interactions

Face-to-face interactions provide an essential ingredient to the development of relationships that are critical to human well-being. Nonverbal cues, such as gaze direction and body language, play a large role in interpersonal exchanges, but they are poorly understood. To provide new insight into nonverbal cues, this paper presents a compact, wearable, real time social behavior monitoring system and details a methodology for extracting a key nonverbal cue called facing time, the amount of time one person faces another during social interactions. The design of an IMU-based embedded device, called Earloop, is discussed along with package design for the ear-mounted wearable. The algorithm developed for real time assessment of facing time is presented, including a unique method employing hysteresis to deal with cases where facing direction windows of different users are overlapping. Test results are presented that demonstrate viability of an IMU-based system for monitoring facing time. Experimental evaluations under balanced, unequal, and disengagement conditions yielded an overall average error of 1.79%, demonstrating high accuracy in real-time monitoring. These results support the concept of using a head-worn wearable to monitor social interactions without the use of cameras and demanding signal processing algorithms. By expanding this system to monitor additional nonverbal cues, this paper lays the groundwork for a future wearable capable of assessing a wide range of social behaviors in real time.