Cross- and multimodal anaphoric references in mystery movies: A cognitive perspective

Abstract The article presents findings from both quantitative and qualitative research regarding the resolution of cross- and multi-modal anaphoric references to objects in two Sherlock Holmes movies directed by Guy Ritchie. The goal is to illustrate how words referring to objects are linked to images depicting the objects as well as sounds produced by the objects to constitute coreferential chains with respect to the viewers’ perception. The theoretical assumptions draw concepts primarily from cognitive linguistics (such as mental spaces, compression, frame-metonymic relations), with additional notions adopted from (verbal) semantics, pragmatics, visual semantics, and film studies. A cognitive account of regular and associative cross- and multimodal anaphora is suggested, along with a model for the representation of cross- and multi-modal coreferential chains for future manual and automated annotation tools. Overall, the work emphasizes the importance of considering both verbal and nonverbal cues in understanding anaphoric references in cinematic contexts by highlighting the cognitive processes involved while viewers integrate information from different modes.