A study on the mechanics, finger movement, and finger function of computer vision technology in guzheng playing posture recognition

Abstract The guzheng is one of the oldest plucked instruments in China, and few studies have been conducted on the guzheng, supplemented by physics. The guzheng playing techniques involving mechanical principles mainly include, “pinching” and “shaking”. The study is based on the 3D-DGR network model to identify the three kinds of 3D dynamic finger postures. Mean shift joint localisation and displacement velocity-based target point position estimation are used to track and identify the finger movements during the playing process. At the same time, the tangent comparison method is used to detect and calculate the finger joint angles, and to judge the finger function of the performer from the angle change when applying the basic playing techniques. The recognition accuracy and the recognition time of the finger movement gesture recognition model constructed in the study are 96% and 1.07ms, respectively, which is a high recognition efficiency. The results of the evaluation of finger function by professionals are consistent with the results of the evaluation using the change of knuckle angle, which proves the recognition accuracy of the computer vision technique.