Keypoint detection in Tai Chi Chuan Essence via Waist and Limbs Feature Separation

Tai Chi Chuan (TCC) teaching process requires face-to-face interaction between learners and masters, which restrains the promotion of TCC since skilled masters are rare and manual evaluation is time-consuming. Skeleton-based action recognition models are effective for this issue, while the existing human skeletal structures and detection frameworks are inadequate for this task. To address these challenges, a novel human skeletal structure aligned with the essence of TCC is proposed, along with a dataset with 2400 annotated images covering fundamental TCC movements. Furthermore, a module named Waist and Limbs Feature Separation (WLFS) is proposed for structured modeling. Based on the spatiotemporal characteristics of TCC movements, the WLFS module explicitly separates keypoints into dynamic and static categories channel-wise. Subsequently, two exclusive GAUs are applied to the static and dynamic regions respectively. This strategy enables the network to learn the distinct features of the separated categories regions, and accelerates the convergence of the network weights during the training process. To preserve the fine-scale features of TCC keypoints in the downsampling, a Multi-Scale Feature Fusion (MSFF) module is integrated into WLFS, which fuses the different spatial resolution feature maps to enhance feature representation of the model at small scales. Experiments on the custom dataset(Tai Chi) and public datasets(MPII, COCO-WholeBody V1.0, Animal Pose, and AP-10K) demonstrate that the proposed method achieves competitive performance and good generalization ability. • Design a novel human skeleton structure. • Construct a tailored dataset consisting of Tai Chi Chuan movements images. • Propose a Waist and Limbs Feature Separation mechanism for keypoint detection.