Block-Wise Variable Conditional Diffusion for High-Fidelity Text-to-4D Generation
Abstract Text-to-4D scene generation is currently impeded by three primary challenges: the prohibitive memory cost of training high-dimensional diffusion models, the semantic misalignment arising from the failure to disentangle static object identity from dynamic motion in text conditioning, and the scarcity of high-quality datasets that accurately correlate subjects, motions, and environments. To surmount these challenges, we present Swin4D , an end-to-end framework for efficient and high-fidelity text-to-4D synthesis. First, we introduce a Block-wise Variable Conditional Diffusion strategy, which mitigates memory constraints and prevents overfitting by dynamically reconfiguring spatial dependencies during training. Second, we propose Global–Local Alignment , a hierarchical guidance mechanism that explicitly disentangles global scene context from local subject-action semantics via separate T5 and CLIP encoders, ensuring spatiotemporal consistency and fine-grained semantic fidelity regardless of prompt length. Finally, we address data limitations by curating RealSynth4D , a large-scale hybrid dataset combining 108k real-world captures with 72k AI-synthesised scenes, establishing robust correlations between subjects, actions, and complex backgrounds to significantly enhance model generalisation.
Authors
- Zhichao Zhang (ORCID: https://orcid.org/0000-0002-7516-059X)
- Ming Xu
- Jinsheng Deng (ORCID: https://orcid.org/0009-0008-2233-7495)
- Hui Chen
Institutions
- National University of Defense Technology (CN)
- PLA Academy of Military Science (CN)
Publication Details
- Journal
- International Journal of Computer Vision
- Published
- 2026-11-13
- DOI
- https://doi.org/10.1007/s11263-026-03119-0
- Primary Topic
- Generative Adversarial Networks and Image Synthesis
- Type
- article
- Field-Weighted Citation Impact
- 0.00
Funders
- National Natural Science Foundation of China
- Science and Technology Innovative Research Team in Higher Educational Institutions of Hunan Province