Color-shape concepts and their representation in macaque monkey

Object concepts are important tools of cognition that often reflect the interaction of a color and a shape. So, “banana” is a yellow crescent. The brain areas that store color-shape interactions are poorly understood. Testing various hypotheses has been challenging because concepts differ between people, and the corresponding likelihood functions and priors about object shapes and colors are not precisely known. Moreover, functional brain patterns differ among individuals. To overcome these challenges, we raised two macaque monkeys to learn about the colors and shapes of a set of 14 objects. Shape was learned faster than color, as in humans. After the monkeys spent four years interacting with the objects, we scanned their brains while they held in mind the color or shape of the objects. We developed a search-light analysis inspired by convolutional networks that is more robust against noise and better generalizes to cross-cue decoding settings. Cross-cue decoding was significant throughout the cortical visual pathway, implying that color-shape concepts are stored in a distributed network. Overall, cross-cue decoding was best in the posterior parcel of inferior-temporal cortex (PIT) (Acc.=.36 +/-.04, chance=.17). Relative to within-cue decoding, cross decoding increased progressively from posterior to anterior inferior-temporal cortex (AIT) and rhinal cortex (r=.86, p=1.2e-16), suggesting the culmination of the ventral visual pathway in AIT/rhinal cortex is a key locus for generating color-shape concepts. Within PIT, color-decoded-from-shape was relatively greater than shape-decoded-from-color, while the opposite pattern was observed within AIT and rhinal cortex. These asymmetries suggest that PIT represents perceptual memory colors, while AIT and rhinal cortex, and their reciprocally connected targets, compute an abstract concept of colors associated with shapes that could be used to guide visual search.