Idiosyncratic facial motions: Uncovering identity information in facial movements through a landmark-based analysis

Previous studies on dynamic faces have shown that both rigid and non-rigid facial movements contribute to identity recognition, and that the identity-specific information inherent in these motions varies depending on the type of facial expression. Therefore, it becomes clear that individuals exhibit distinct idiosyncratic patterns in their facial movements, which can serve as cues towards their identity. However, the specific features of facial movements that contribute to this uniqueness remain unclear. Here, we employed machine learning techniques to measure and quantify motion information in facial expressions, using a dataset of six basic emotional facial expressions (anger, disgust, fear, joy, sad, surprise) performed by 12 German and 12 Turkish lay actors. An automated facial landmark detection tool was applied to measure the positional changes of landmarks at the peak of each expression relative to a neutral baseline. We then trained a Linear Discriminant Analysis (LDA) model with these landmark shifts to classify the emotional expressions. This first LDA model was able to classify the type of emotional expression (Accuracy: 44%, p<0.001), independent of the actor’s identity. More strikingly, another LDA model, trained to classify the identities of the 24 actors across different expressions, successfully predicted their identity (Accuracy: 45%, p<0.001). Furthermore, the landmark positional changes provided useful information for classifying actors’ gender (Accuracy: 59%, p<0.01) and country-of-origin (Accuracy: 71%, p<0.001), supporting previous studies on cultural and gender-based variations in facial expressions. In conclusion, our study shows the richness of information embedded in facial motion features, extending beyond emotional expression to contain aspects of the actor’s identity, gender, and cultural background. This landmark-based approach emerges as a promising tool to unravel the nuances of idiosyncrasies in facial movements, offering valuable insights into the intricate interplay of expression, identity, and cultural factors.