Exploring pose estimation in instrumental composition: the Body Fragmented project

Introduction In contemporary and experimental music composition, the integration of physical movement via sensors and digital technology offers new pathways for music composition, performance and interdisciplinary practice. This research explores how pose estimation can be used to generate movement-based notation and support embodied musical expression. Methods Through the Body Fragmented composition project, this study introduces a methodology that centres the composer and performer’s bodies in instrumental composition using pose estimation technology. The approach supports non-linear collaborative processes between composer and performer, facilitating a movement-led instrumental composition practice. A guiding question “what does that movement express and how could that sound” provided a conceptual anchor and encouraged an openness to the interpretation of the pose estimation outputs, allowing it to suggest sonic material and musical expression. Results Findings reveal that pose estimation can effectively capture expressive movement for compositional development, support performer interpretation through visual scores, and enhance collaborative dialogue. The study also identifies limitations in pose estimation’s ability to convey nuanced musical gestures, and detailed musical information, prompting the integration of supplementary notation materials. Discussion The reflective methodology enabled the collaboration to explore new forms of movement-led methodologies in instrumental composition.