Dream4D: Lifting Camera‐Controlled I2V towards Spatiotemporally Consistent 4D Generation
Abstract Generating spatiotemporally consistent 4D content requires joint modeling of high‐fidelity spatial details and realistic temporal dynamics. However, existing methods typically lack world knowledge learned from expansive 4D observations. As a result, generated scenes often struggle to maintain pose and temporal consistency in complex dynamic environments, leading to degraded generation quality. To address these challenges, we propose Dream4D , a novel 4D generation framework that integrates spatiotemporal priors from video generation with explicit sequential camera control. Our method adopts a three‐stage pipeline. We first employ a vision–language model (VLM) to determine the camera trajectory that best captures the intended view from the input image and the instruction text. Next, a pose‐conditioned diffusion model generates geometrically consistent image sequences along this trajectory. Finally, we use an advanced 4D generator to reconstruct a coherent and persistent 4D representation conditioned on the generated video. Dream4D jointly leverages the rich temporal priors of video diffusion models and the geometric awareness of reconstruction models to achieve high‐quality image‐to‐4D generation. Extensive experiments demonstrate that our approach significantly outperforms existing methods in both pose and temporal consistency.
Authors
- Kun-Fang Li (ORCID: https://orcid.org/0000-0001-6204-4074)
- Xiaoyan Liu (ORCID: https://orcid.org/0000-0002-6442-7893)
- Yuehao Song (ORCID: https://orcid.org/0009-0003-2711-944X)
- Jiaxin Liu
Institutions
- Hong Kong Polytechnic University (HK)
- Chinese University of Hong Kong (HK)
- UNSW Sydney (AU)
- Huazhong University of Science and Technology (CN)
Publication Details
- Journal
- Computer Graphics Forum
- Published
- 2026-11-14
- DOI
- https://doi.org/10.1111/cgf.70617
- Citations
- 1
- Primary Topic
- Generative Adversarial Networks and Image Synthesis
- Type
- article
- Field-Weighted Citation Impact
- 1.46