Towards a novel conceptualization of prosody that accounts for spoken and visual signals

Abstract Prosody influences speech organization by signaling phrasal prominence, grouping patterns, and speakers’ pragmatic intentions. While traditionally viewed as restricted to speech, research shows prosody is also conveyed visually. This article reviews research showing strong parallels between spoken prosody and co-speech gestures in prominence marking, grouping phrasal structures, and signaling pragmatic intent. We extend this discussion and propose a modality-neutral prosodic framework hypothesis comprising three propositions: (a) prosody should be viewed as a modality-neutral grammar component that operates as an abstract level of representation while adapting to different sensory channels and language modalities; (b) in spoken languages, prosody is implemented flexibly through two distinct channels, spoken and gestural, which enable to mark prominence, grouping and meaning in a multimodal way; (c) parallel implementations are found in the way prosody is manifested in spoken and sign languages. A modality-neutral view of prosody will enrich current formal and developmental theories of language.