Temporal segmentation principles in vision and audition

Our sensory system automatically and unconsciously extracts patterns at varying complexity from the environment and these patterns bias sensitivity to changes, accuracy during recognition and also guide attention. A growing body of evidence shows that sensory integration starts at low-level cortical areas as neural correlates of temporal integration (chunking) are already present in the primary sensory cortices. Based on this, we hypothesized that certain chunking principles and the resulting perceptual biases are similar across modalities. To test our hypothesis, we focused on a well-known auditory chunking principle called the Iambic-Trochaic law (ITL). Established in language processing, ITL says that longer syllables in a sequence signal word ends and similarly, a longer duration of a tone is understood as a closing element of a perceptual chunk. ITL is considered a basic law since it has been found across multiple species, but it is unknown whether it generalizes across modalities. We implemented a Short-Short-Long stream segregation go/no-go paradigm for human participants in an identical manner in the visual and auditory modalities and found that although the general performance was lower in vision, biases in sensitivity to deviations showed the same bias in the two domains (N=17). Extending the paradigm to a Short-Long-Long stream, we replicated the basic effect while clarifying that this sensitivity bias could neither be explained by the repetition of individual elements nor by the absolute duration of the individual elements alone in either sensory domain. Instead, this bias depended on an unconscious chunking process that integrated effects related to both duration of the individual elements and the internal structure of the stream such as the number and arrangements of the long elements. Our results support the existence of domain-general non-linguistic grouping principles incorporating structural features of ever-increasing complexity that could ultimately give rise to high-level chunks leading to object concepts.