Evaluation of piano performance based on motion analysis using inertial sensors

In recent years, fueled by both the improvement of communication technology and societal changes fueled by the COVID-19 pandemic, the demand for various online classes and courses has increased, especially for physical education and music lessons. However, current online lessons are limited to instruction through screens, an inadequate approach that lacks precision and efficiency. For more efficient teaching, this study measured upper body movements using inertial sensors, which are inexpensive and have few spatial limitations, and focused on piano playing because it requires high precision for measurement movements. We calculated a performance technique called tremolo, in which two notes separated by the pinky and the thumb are quickly and alternately repeated. A wearable hand-movement-measurement system using an inertial sensor was attached to one skilled subject and several novices, and a movement model was constructed to obtain each joint’s angle. Singular value decomposition was performed on the obtained joint angles of the hand and the arm to evaluate the characteristics of the movements, and we investigated how the tremolo of the novices changed when their learning was based on the characteristics of the skilled subject. Every novice’s movement results approached the movements of the skilled subject before and after the teaching, and the order of the improvement rates of the movements of each subject was consistent with the order of the improvement rate of their tremolo skills. By analyzing and extracting the characteristics of the skilled subject’s tremolo, we confirmed that the novices learned the skilled subject’s tremolo and improved their own skills.