Social context shapes facial dynamics: human and machine decoding of conversation topics

Abstract Conversation is fundamental to the human species, with facial expressions playing a crucial role in establishing shared understanding within specific conversational contexts. We hypothesized that variations in conversation topics, differing in levels of tension and personal disclosure, would elicit distinct facial behavior dynamics. We assessed facial activity during two types of natural conversations with varying tension levels in triads of unacquainted individuals: "get-to-know-each-other" and “moral dilemma” discussions. Human observers classified the conversation type with 82.11% accuracy based on facial dynamics alone. Strikingly, a machine learning model using three facial action units (AUs 4, 6, and 12) during speech-free moments achieved comparable accuracy (82.14%). Further analyses revealed that the model relied on the temporal dynamics of these AUs to distinguish conversational contexts. These findings show that machine-based facial coding, coupled with deep learning, can infer conversational context from facial expressions, offering a scalable tool for analyzing natural social interaction.