Self-supervised multiview reconstruction of eyeglasses frames with thin structures

Using traditional depth and image-based methods for reconstructing eyeglass frames poses significant challenges due to their unique properties, such as minimal textural features and thin components. In this paper, we propose a novel self-supervised approach for reconstructing eyeglass frames based on vertex displacement prediction. This method enables the precise 3D shape recovery of full-frame eyeglass models from multi-view images. To begin, we first establish an eyeglasses frame template, leveraging prior and domain-specific knowledge. Subsequently, when provided with an input image of an eyeglass frame characterized by thin structures and few texture features, we develop an iterative self-supervised camera pose estimation module to accurately predict the camera pose. Following this, we employ a self-supervised vertex offset prediction network to extract features from both the input multiview images and the 3D template eyeglass frame model. This network predicts the vertex displacement of the frame to be reconstructed relative to the template frame model. We iteratively optimize the network to obtain precise vertex offsets by minimizing the image loss between rendered images and input images using differentiable rendering. Experimental findings from both synthetic and real datasets demonstrate the effectiveness of our proposed method.