The multidimensional representation of facial attributes.

As primates, our social behaviour is shaped by our ability to read the faces of the people around us. Our current understanding of the neural processes governing ‘face reading’ comes primarily from studies that have focused on the recognition of facial expressions. However, these studies have often used staged facial expressions, potentially disconnecting facial morphology from genuine emotion and circumstance. Therefore, a reliance on staged stimuli might be obscuring our understanding of how faces are perceived and recognised during everyday life. Here our goal was to identify the core dimensions underlying the mental representation of expressive facial stimuli using a data driven approach. In two behavioural experiments (Experiment 1, N = 940; Experiment 2, N = 489), we used an odd-one-out task to measure perceived dissimilarity within two sets of faces; 900 highly-variable, naturalistic, expressive stimuli from the Wild Faces Database (Long, Peluso, et al., 2023 Sci Reports, 13: 5383) and 670 highly-controlled, staged stimuli from the NimStim database (Tottenham, Tanaka, et al., 2009 Psychiatry Res, 168: 3). Using Representational Similarity Analysis, we mapped the representation of the faces in the Wild and NimStim databases, separately, and compared these representations to behavioral and computational models. We also employed the state-of-the-art VICE model (Muttenthaler, Zheng, et al., 2022 Adv Neural Inf Process Syst) to uncover the dimensions that best explained behaviour towards each of the face sets. Collectively, these results indicate that the representation of the Wild Faces was best characterised by perceived social categories, such as gender, and emotional valence. By comparison, facial expression category explained more of the perceived dissimilarity among the NimStim faces than the Wild Faces. These findings underscore the importance of stimulus selection in visual cognition research and suggest that, under naturalistic circumstances, humans spontaneously use information about both social category and expression to evaluate faces.