Are Abstract Relational Roles Encoded Visually? Evidence from Priming Effects

Whereas it is well established that the visual system encodes comparative relations such as Longer and spatial relations such as Above, it remains controversial whether it encodes abstract relational roles such as Agent and Patient in visual events. The present experiment tested whether the latency of a color localization task is affected by the abstract role bindings of a preceding event. Method: Each trial included a static target image preceded either by a brief silent video of a priming event (on Session 1) or by an audio-visual presentation of an English sentence describing the same event (on Session 2). Example sentence: 'The red goat on the left knocked down the blue goat on the right." There were 64 videos counterbalancing 4 event types (launching, deforming, breaking, and a relationally ambiguous control), 4 animal species, 2 role bindings (Agent is red vs blue), and 2 locations (red on the left vs right). The set of static targets were the final frames of the same videos. The role bindings were either repeated, switched, or ambiguous across the target and prime, counterbalanced across trials. The mask-enforced presentation duration of the target was calibrated individually to ensure 90% accuracy of the color localization task. Dependent variable is latency on correct trials. 15 participants x 2 sessions x 768 trials/session. Results: Whereas the role bindings of the linguistic primes had no statistically significant effect on the latency of the visual task, the role bindings of the video primes did have an effect: The latency on unambiguous trials (on which both target and prime had well-defined Agents and Patients) was significantly greater than that on ambiguous trials (on which at least one component lacked clear relational roles). This is evidence that the visual system is sensitive to (the ambiguity of) the role bindings of abstract relations.