Human pose feature enhancement for human anomaly detection and tracking

Abstract Human pose, represented as a set of keypoints, is a principal feature in pose-based human anomaly detection and tracking. However, using keypoint alone for both tasks encounter loss during heavy occlusion or missed keypoint detection, which leads to lower the model’s performance. To address these challenges, we propose a method that employs multi-object tracking as the tracker, incorporating human pose estimation to maintain robust tracking even when keypoint detection fails. Additionally, we introduce a pose selection module that selects the most appropriate pose and recovers the incomplete pose of each individual target. Accurately determining the most representative pose of each individual is crucial, as it enhances the precision of activity recognition and improves anomaly detection accuracy. Our pose selection module leverages various pose estimation models to generate diverse pose candidates for each tracked object, and then the similarity scores between those poses are computed to identify the most significant one. Our approach demonstrates improved performance, achieving an accuracy of up to 86.4%, surpassing state-of-the-art methods.