The social self: integration of multimodal communication signals in dyadic interactions

Abstract Humans are in continuous exchange with other persons, thereby integrating the individual self and the social self. This requires the production of different verbal and nonverbal signals at the subpersonal level (individual self) and, at the same time, their coordination with various perceived signals at the interpersonal level (social self). Many of our communicative signals are conveyed substantially nonverbally—that is automatically, intuitively and effortlessly, requiring dynamic coordination over time (‘chronemics’). Whereas successful communication is usually perceived as pleasant and satisfying, communication can also be experienced as disrupted if chronemics do not unfold properly. With respect to the particularly informative example of autism spectrum disorder, we propose the concept of ‘double invisibility’. Non-autistic persons intuitively produce and expect a rich, multimodal stream of communication signals without being aware of it (first invisibility). By contrast, autistic persons produce and expect nonverbal cues either to a significantly lesser extent or not at all without being aware of it (second invisibility). This mismatch leads to misunderstandings and disturbed communication. The coordination within the dyad that is continuously constructed through real-time communication is crucial to understand the full complexity of multimodal interactions and their disturbances. This article is part of the discussion meeting issue ‘From being to identity: analysis and synthesis of the self’.