View-symmetric representations of faces in human and artificial neural networks

Models of face processing propose that the neural representation of face identity is initially view-specific, but then becomes view-invariant to enable recognition across different images. Recent studies have suggested that view-symmetry may be an important intermediate representation between view-specific and view-invariant representations. In this study, we compared view-symmetry in humans and a deep convolutional neural network (DCNN) trained to recognize faces (VGG-Face). First, we asked whether view-symmetry is an emergent property of the DCNN for different rotations of the head. We compared the output in the early convolutional layers and the later fully-connected layers of the DCNN to changes in viewpoint caused rotations in yaw (left-right), pitch (up-down) and roll (in-plane rotation). We found that there was an initial view-specific representation in the convolutional layers for yaw, but that a view-symmetric representation emerged in the fully-connected layers. We also found that the ability to differentiate identity was greater across symmetrical compared to non-symmetrical viewpoints. In contrast, we did not find a similar transition from view-specific to view-symmetric representations for either pitch or roll. Next, we compared patterns of response in the DCNN to changes in viewpoint for yaw with corresponding behavioural and neural responses in humans. We found that responses in the fully-connected layers of the DCNN correlated with judgements of perceptual similarity. We also found that the response of the convolutional layers of the DCNN correlated with responses in early visual areas, but that the response of the fully-connected layers correlated with responses in higher visual regions. These findings suggest that view-symmetry emerges when opposite rotations lead to mirror images. The difference in the response to same identity and different identity faces suggests that view-symmetric representations may be important for the recognition of faces in humans and artificial neural networks.