GSAlign: Optical Alignment for Photorealistic Dynamic 3D Reconstruction from UGC in Web3
User-generated 3D content plays a crucial role in enabling truly immersive experiences within the Web3 metaverse. Although significant progress has been made in dynamic 3D content creation and animation using multi-camera setups, they typically require professional equipment and strict synchronization. In contrast, a dynamic and inclusive Web3 ecosystem is collectively built by digital assets from anyone, at any time, and from anywhere. Most Web3 participants are therefore decentralized and non-professional, and their contributed videos, even of the same scene, are usually captured asynchronously using handheld devices. In this paper, we present GSAlign, a novel framework that achieves high-fidelity dynamic 3D reconstruction from unstructured, user-generated videos in the Web3 metaverse. To address the temporal and spatial misalignments inherent in such data, GSAlign integrates three key modules: Epipolar-guided Temporal Alignment (ETA), Motion-centric Separated Reconstruction (MSR), and Background-guided Global Pose Alignment (BGPA). We detail the design of each module and their integration toward a practical end-to-end implementation. Our evaluation of GSAlign on real-world user-generated videos demonstrates robust reconstruction of dynamic 3D scenes despite unsynchronized captures, sparse views, and handheld camera motion.
Authors
- Yili Jin (ORCID: https://orcid.org/0000-0002-7127-8902)
- Jiangchuan Liu (ORCID: https://orcid.org/0000-0001-6592-1984)
- Xue Liu
- Rui Qian
- Daheng Yin (ORCID: https://orcid.org/0000-0002-9431-4240)
- Isaac Ding
Institutions
- Simon Fraser University (CA)
- Mohamed bin Zayed University of Artificial Intelligence (AE)
Publication Details
- Journal
- ACM Transactions on Multimedia Computing Communications and Applications
- Published
- 2026-06-09
- DOI
- https://doi.org/10.1145/3820054
- Primary Topic
- Virtual Reality Applications and Impacts
- Type
- article
- Field-Weighted Citation Impact
- 0.00