Cortical representations of core visual material dimensions

In every waking moment, we perceive numerous visual materials from the objects, surfaces and environment around us. How does the brain represent the great diversity of materials and their properties? We recently addressed the mental representation of materials using a novel dataset consisting of 600 images spanning 200 material categories (the STUFF dataset), by crowdsourcing over 1.8 million material similarity judgments. This revealed 36 core dimensions that capture similarity relationships between materials (Schmidt, Hebart, Schmid & Fleming, 2023). To determine the neural representation of these dimensions in the human brain, here we acquired a densely-sampled functional MRI dataset using these images, which we paired with an encoding model of the 36 material dimensions. Each of the 600 images was presented to six participants 14 times each across multiple scanning sessions. The whole brain activation map of each material dimension was then obtained by modeling the dimension score of each image in each of the 36 dimensions (Schmidt et al, 2023). Comparing the voxel-wise activation intensity across material dimensions revealed superimposed cortical maps associated with each of the dimensions. We found that dimensions related to the fine scale granularity of the material are particularly represented in early visual areas (V1-V3). In contrast, dimensions related to hard shapes preferably activated lateral occipital (LO) cortex, indicating a dichotomy between cortical regions associated with shape and fine texture. Flexible and soft material dimensions exhibited particularly strong responses in area hMT+/V5, suggesting that motion sensitive regions also encode the capacity of materials to deform. Finally, color dimensions, which span diverse material categories, were represented less consistently across participants, suggesting that material properties might actually be a more consistent organizing principle than color. Together, our findings provide a comprehensive mapping of material representations across cortical regions in the human brain.