Sequential perception of tone and focus in parallel–A computational simulation

• Model simulations can match human perception of tone and focus from f 0 contours. • Tone can be recognized without knowledge of focus, and vice versa. • Focus can be recognized syllable by syllable based on probabilistic inference. • Humans tend to hear neutral rather than broad focus as default. • There is no advantage of filtering out variability through f 0 normalization. Speech is produced continuously over time. So, the information it conveys, including intonational functions, also unfolds over time. But many intonational functions are encoded across whole utterances rather than only within certain words. How can perception process speech signals continuously over time, even for communicative functions that are globally encoded? In this study we used computational simulation to test the idea that even for intonational functions with large temporal scopes, it is possible to process f 0 contours syllable-by-syllable, and recognize the functions by continuous estimation of progressive probabilistic inference. We trained SVM and GRU models to simulate the perception of Mandarin tone and sentence focus with either syllable-sized or sentence-sized f 0 contours as input. The sentence-wide f 0 contours are gated at different syllable locations to test the incrementality of the recognition of tone and intonation. We also tested human listeners’ perception of tone and focus with full and fragmented f 0 contours from the same dataset to evaluate the validity of the simulated perception. The results showed that the simulated syllable-by-syllable processing of tone and focus generated the closest recognition patterns to human perception. The simulations also show that there is little difference whether tone and focus are recognized separately or as tone-focus combinations, which suggests that despite sharing the same acoustic dimension, the two functions are sufficiently separated from each other in their f 0 coding.