Serial Dependence operates on categorical rather than physical representations: evidence from behavior and EEG

ABSTRACT Our environment is constantly changing but our perception is stable. This stability may result from the integration of successive visual images that smooths over spurious variations in input. Such a process manifests in a behavioral phenomenon called Serial Dependence (SD), an attractive bias between successive stimuli. Here we questioned whether SD originates from physical stimulus representations or at a higher level, that of categorical perception. We constructed face morphs from three prototypes of the same face expressing anger, fear and sadness. We measured the perceived similarity between each pair of morphs with an odd-one-out experiment, to build a behavioral representational similarity matrix. Emotional expressions were perceived categorically. Subjects then reproduced the perceived facial expression of successively presented morphs by adjusting a response cue while their cerebral activity was recorded with EEG. We showed attractive serial dependence tuned to perceptual distance rather than physical similarity. We used representational similarity analysis to discover the geometry of the mental representation of faces, to capture both the information represented in a neural signal and its format. EEG representational similarity analyses showed that neural responses encoded emotional ambiguity rather than category membership, with near-prototypical faces eliciting similar patterns across emotions. Models combining feature tuning with categorical structure best accounted for behavioral biases. Neural signals also carried traces of past stimuli. Stimuli for which memory-based representations diverged most from sensory representations showed the strongest attractive serial dependence.