Combined conceptual and perceptual control of visual attention in search for real-world objects

When we search for an object in the natural visual environment, we sometimes know exactly what the object looks like. At other times, however, we know only the category of the object. For example, if we are looking for our own bath towel, we might know that it is brown and is folded into a rectangle. However, if we are looking for a towel in a friend's house, we might not know its color or whether it is folded or lying in a clump. Consequently, we may sometimes be able to use specific perceptual features to guide search, but some search tasks are so conceptual in nature that the relevant visual features are difficult to specify. Here, we found that eye-movement patterns during visual search could be predicted by perceptual dimensions derived from crowd-sourced data (THINGS), but only when observers had previously seen the specific target object. When only the category of the desired object was known (because the observer had never seen the specific target), eye-movement patterns were predicted by conceptual dimensions derived from a natural language processing model (ConceptNet), and perceptual features had no significant predictive ability once the conceptual information was statistically controlled. In addition, as observers gained experience searching for a specific exemplar of a category, they became progressively more reliant on perceptual features and less reliant on conceptual features. Together, these findings provide novel evidence that conceptual information can influence search, especially when the precise perceptual features of an object are unknown.