Foreground-Driven Contrastive Learning for Unsupervised Human Keypoint Detection

Human keypoint detection has significant value in computer vision tasks such as human-machine interaction. Recently, unsupervised human keypoint detection has become prevalent due to concerns about data privacy. Most existing methods are based on a reconstruction process that extracts appearance and pose information from transformed image pairs and spatially aligns them to obtain a reconstructed image for detection. However, these methods suffer from an issue because they reconstruct the entire image, which can easily lead to some keypoints being assigned to the background region. In this work, we believe that focusing on independent reconstruction and detection of the foreground region can mitigate the above issue. To this end, we propose a novel unsupervised human keypoint detection scheme to achieve reliable detection, which focuses on reconstructing and detecting keypoints in the foreground. Specifically, we first use a segmentor to separate the foreground and background of the image, for reconstruction and detection to be done only on the foreground region. Considering that keypoints vary due to changes in appearance and pose, we then introduce the contrastive loss to expand the feature space and enhance the network's robustness. Depending on the insertion position of the segmentor, we differentiate the proposed scheme into two versions: the effective version and the efficient version. Experimental results on popular datasets show that the proposed method exhibits superior performance. Specifically, on the BBC Pose dataset, the effective version achieves a <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\mathbf{7.0\%}$</tex-math></inline-formula> performance improvement. The efficient version leads to a <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\mathbf{5.7\%}$</tex-math></inline-formula> performance enhancement without sacrificing the inference speed.