An Improved Zero-1-to-3 3D Reconstruction Method from a Single Image

To improve the consistency of image modifications and multi-view images while optimizing the use of stable diffusion resources, an improved Zero-l-to-3 method is proposed. This approach generates 3D consistent multi-view images by altering the camera angle of an object based on a single RGB image. The self-attention approach, which operates the de-noising U-Net model on the reference image, supplies suitable local adjustment inputs, processes noise in the image adjustments, and produces an image that retains similar textures and semantic information as the reference image. Subsequently, a tile layout strategy is applied to model the joint distribution across multiple images. Ultimately, the SJC method is applied for <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$3\mathrm{D}$</tex> reconstruction. Experimental results indicate that this approach outperforms the comparison methods in terms of both objective metrics and subjective visual quality. It is capable of producing high-quality, consistent multi-view images from a single input image, addressing issues of texture loss and geometric misalignment, while ensuring the image generation quality of the stable diffusion model.