Structural unity of audio–visual experiences

Abstract The temporal unity of multimodal audio–visual experiences seems to be stronger than their spatial unity. In particular, when one has an ordinary audio–visual experience, one is able to recognize that there is a non-visual part of space—behind one's head—but one is not aware of purely visual or auditory parts of time. This paper investigates the spatiotemporal unity of audio–visual experiences by applying a distinction between experiential contents and experiential structures, that is, relatively invariant ways of organizing content. General structural factors that determine multimodal unity are identified, which are also likely to be relevant to other types of multimodal experiences. In particular, it is shown that certain relationships between so-called ‘limitation structural boundaries’ lead to a limited multimodal unity where one is aware of unimodal parts of a multimodal experiential structure.