Hierarchical Structured Latents for Efficient 3D Generation

Generating high-fidelity 3D geometry from a single image remains a challenge due to the complex trade-off between representation capacity and computational efficiency. Existing methods often employ uniform latent distributions, which lack the flexibility to allocate resources according to local geometric complexity. In this paper, we utilize a two-stage framework for image-conditioned 3D generation based on a multi-scale latent representation. Our approach decomposes the generation process into the creation of a multi-scale spatial structure followed by the synthesis of fine-grained surface details. To ensure structural integrity, we introduce a generation-friendly octree reparameterization that utilizes a top-down masking rule to guarantee spatial exclusivity and completeness by construction, effectively bypassing the need for generative models to learn hard structural constraints. Furthermore, we propose Scale-Modulated Positional Embedding (SMPE), which adaptively adjusts the frequency bandwidth of positional encodings based on the local latent scale. This allows transformers to dynamically aggregate global context for coarse regions and localized details for complex areas. Extensive experiments demonstrate that our method achieves superior reconstruction fidelity and generation quality, offering a more efficient and adaptive solution for high-resolution 3D asset creation. Code for this paper is at https://github.com/XiongYX0817/HSL-3D.