Boundaries of gestures: Naive segmentation of the stream of human hand movements

Naïve observers and researchers alike segment behavior into manageable units for analysis; yet, the nature and application of these units vary. This study examines differences in gesture segmentation between naïve observers and experts, as well as variability among naïve observers. We review a range of gesture definitions and segmentation cues, emphasizing the distinctions between perceiving gestures in interpersonal interactions and identifying them in video recordings, whether by trained or untrained annotators. In our empirical study, naïve observers annotated gestures and their prominent phases under two conditions: video-only (“silence”) and video with audio (“speech”), using ELAN software. The analyzed material consisted of an excerpt from a Member of Parliament's address. The results revealed no significant differences between conditions in the number of identified gestures, total or mean gesture duration. However, we found different levels of inter-annotator agreement in the two conditions and identified certain patterns in how participants distinguished gestures from the continuous flow of hand movements, which differed from expert segmentations. Results showed variability in annotation granularity, with naïve observers marking fewer gestures than experts. Additionally, prominence marking varied: naïve observers differed from experts, especially in the “silence” condition. These findings underscore the nonintuitive nature of gestures as behavioral units and the influence of theoretical training on segmentation practices. • The study reveals the complexity and traps of gesture annotation. • Untrained participants annotated gestures with and without co-occurring speech. • Untrained annotators distinguish significantly less gestures than experts. • Untrained annotators' understanding of gesture is close to Gesture Unit. • The presence of speech does not increase inter-annotator coherence.