Metaverse-driven instrumental music teaching: a human-computer interactive perception method

Teaching musical instruments traditionally requires one-on-one instruction, creating challenges of accessibility, cost and feedback quality. Metaverse technologies offer promising solutions but face limitations in gesture recognition accuracy and haptic feedback realism. This paper proposes a metaverse-driven human-computer interaction method for instrumental music teaching that addresses these challenges. Through analysis of similar gestures in instrumental music performance, we discovered that inflection point count and gesture displacement vector angle significantly improve discrimination between similar instrumental gestures. By incorporating these features into our dynamic gesture recognition method, we enhance identification accuracy for specialised musical movements. Additionally, we simulate the haptic interaction process between virtual hands and instruments to achieve realistic performance effects and synchronous multi-channel sensory feedback in real-time interactions. Our approach focuses on six degrees of freedom haptic interactions during multi-point contact scenarios. Experimental results demonstrate that our method achieves higher recognition accuracy than baseline approaches for customised gestures and instrument-specific movements. This research contributes to more accessible, effective and immersive instrumental music education in virtual environments, potentially democratising access to quality music instruction regardless of geographical or economic constraints.