A Novel 3D Keypoint Detection Method based on Self-supervised Learning

3D keypoint detection has a wide range of applications in artificial intelligence, such as 3D reconstruction, autonomous driving, and robot navigation. 3D keypoint detection based on self-supervised learning does not require labeled data, but at the same time, the lack of guidance from labels makes this type of approach somewhat restrictive. In this work, we propose a novel self-supervised 3D keypoint detection method that employs an encoder-decoder architecture. In the encoder stage, to better capture the contextual information around the keypoints, we adopt the combination of conventional 3D graphics rendering and SoftRasterizer rendering to obtain the multi-view patches of each point in the point cloud, and introduce a local attention mechanism to extract local features, and finally through a multi-view feature fusion network to obtain efficient and robust point cloud descriptors. In the decoder stage, we propose a fine secondary reconstruction network to generate fine-grained point clouds after coarse reconstruction to facilitate more accurate keypoint localization. We conduct experiments on KeypointNet and SMPL datasets for keypoint detection and 3Dmatch and ETH datasets for the downstream geometric registration task. The experimental results show that the proposed method not only detects accurate keypoints, but also has strong stability and generalization.