Gesture Flux: Unveiling Budaixi Hand Gestures through Mixed Reality Performance 29

This paper presents Gesture Flux, an immersive performance system that integrates mixed reality (MR) gesture recognition, Budaixi (traditional Taiwanese glove puppetry), and contemporary dance. Inspired by established hand gestures from Budaixi performance, the work focuses on digital reinterpretation, using MR as a medium for movement and spatial mediation. It explores how hand gestures traditionally hidden beneath the puppet body can be brought into perceptual awareness through embodied interaction. The system combines real-time hand-gesture tracking with particle-based visual effects, transforming the dancer’s hand movements into luminous trajectories in space. Three interactive modes—Basic, Extended, and Radiant—are designed to represent different stages of gestural expression, ranging from direct gesture articulation to perceptual extension and energetic radiation. Through these modes, the work engages with the embodied and performative potential of Budaixi gestures within hybrid physical–virtual environments. Across four public performances, the dancer performs while wearing an MR headset, with Budaixi cultural gestures rendered as particle-based visuals through hand movements within a shared immersive space. This enables Budaixi gestures to be re-generated and perceived, and suggests the system’s potential as a medium for cultural reinterpretation on the contemporary stage.