Decoding Contextual Effects in Vision: A Cross-Species Behavioral Approach

The significance of context in visual perception is undeniable. Our understanding of the natural world is shaped not just by the foveated visual objects but also by the surrounding scene and prior experiences. While the influence of context on vision has been demonstrated psychophysically, the underlying mechanisms integrating objects and surrounding information during scene comprehension are not fully understood. Studies have extensively examined "low-level" contextual effects, such as extra-classical receptive fields and surround suppression, yet gaps remain in comprehending how context affects "higher-level" visual recognition tasks. To elucidate these neural processes, a detailed examination of the neural networks involved is essential. Rhesus macaques, with their visual processing circuits akin to humans, present an ideal model for this purpose. In our study, we assessed the behavior of 90 human participants via Amazon Mechanical Turk in a binary match to sample object discrimination task, using images with varied contexts (full, incongruent, no context, etc.). The results revealed a significant alteration in human performance due to contextual changes, exhibiting a consistent behavioral pattern across context categories (trial-split reliability of ~0.8). This finding was crucial for comparison with macaques. After training monkeys (n=2) to achieve ≥80% accuracy in object categorization with full-context images, we exposed them to the same contextually manipulated images. The behavioral variance shared between humans and monkeys was significant (~31%), and not attributable to low-level image factors such as object size or contrast. Interestingly, naive macaque inferior temporal (IT) neural responses did not fully account for the observed human-monkey shared variance (13% of image-level explained shared variance), suggesting that the effects are likely driven more by learning processes and feedback mechanisms than by innate IT response statistics. This research paves the way for future investigations into the neural mechanisms of contextual modulation in primate vision.