The Gesture Homunculus: A Body-Cortical Topography of User-Defined Gestures

Gesture research for interactive computer systems has primarily relied on taxonomies using external dimensions of gesture articulation. While effective for documenting gestures and supporting computational recognition, this focus provides an incomplete account by overlooking internal dimensions related to how the body parts involved in gesture articulation are represented in the brain. To address this aspect, we introduce the Gesture Homunculus, a body-cortical topographical representation grounded in the medical literature, reinterpreted through the lens of user-defined gestures. To construct it, we analyzed 2,294 elicited gestures and 3,637 corresponding body parts, which we relate to somatosensory and motor cortical mappings. Based on this representation, we identify future work opportunities for gesture elicitation studies, such as exploration of underrepresented body parts, combination of multiple body parts within a single gesture, and designing gesture interactions that span both the somatosensory and motor cortices.