Using texture synthesis to identify the features supporting coarse and fine object categorization

A key question within visual cognition is how we determine an object’s category, and which intermediate visual features contribute most strongly to this process. Past work indicates that low-level and mid-level features may be sufficient to support certain types of object category judgments even in the absence of high-level features. However, it is not known how the role of these features differs depending on the granularity of the categories. One possibility is that coarse categories may be detectable on the basis of simpler features, while finer-grained category distinctions might require more complex features to discriminate. Here, we tested this hypothesis behaviorally by leveraging a computational texture synthesis tool (Gatys, Ecker, & Bethge; 2015, NeurIPS), which synthesizes images that maintain the local textural statistics of a target natural image. We generated versions of target object images that matched the statistics of their representations at multiple layers of a deep neural network, resulting in images that continuously vary in their feature complexity. In a series of online experiments human participants judged - at either the coarse (superordinate) or fine (basic) level - the category of the manipulated images presented in either grayscale or color. Our results show that while category discriminability improves for more complex images, some categories – particularly natural categories such as “insect” and “vegetable” – can be discriminated at above chance levels even from the simplest texturized images. Consistent with our hypothesis, performance with the simplest images was higher for coarse versus fine categorization, and varied across individual object categories within the experiment. These results indicate that features at multiple levels of complexity may contribute to object categorization.