Block-Wise Variable Conditional Diffusion for High-Fidelity Text-to-4D Generation

Abstract Text-to-4D scene generation is currently impeded by three primary challenges: the prohibitive memory cost of training high-dimensional diffusion models, the semantic misalignment arising from the failure to disentangle static object identity from dynamic motion in text conditioning, and the scarcity of high-quality datasets that accurately correlate subjects, motions, and environments. To surmount these challenges, we present Swin4D , an end-to-end framework for efficient and high-fidelity text-to-4D synthesis. First, we introduce a Block-wise Variable Conditional Diffusion strategy, which mitigates memory constraints and prevents overfitting by dynamically reconfiguring spatial dependencies during training. Second, we propose Global–Local Alignment , a hierarchical guidance mechanism that explicitly disentangles global scene context from local subject-action semantics via separate T5 and CLIP encoders, ensuring spatiotemporal consistency and fine-grained semantic fidelity regardless of prompt length. Finally, we address data limitations by curating RealSynth4D , a large-scale hybrid dataset combining 108k real-world captures with 72k AI-synthesised scenes, establishing robust correlations between subjects, actions, and complex backgrounds to significantly enhance model generalisation.

Authors

Institutions

Publication Details

Journal
International Journal of Computer Vision
Published
2026-11-13
DOI
https://doi.org/10.1007/s11263-026-03119-0
Primary Topic
Generative Adversarial Networks and Image Synthesis
Type
article
Field-Weighted Citation Impact
0.00

Funders

Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Block-Wise Variable Conditional Diffusion for High-Fidelity Text-to-4D Generation

Zhichao Zhang, Ming Xu, Jinsheng Deng, Hui Chen
International Journal of Computer Vision
Generative Adversarial Networks and Image Synthesis
article

Block-Wise Variable Conditional Diffusion for High-Fidelity Text-to-4D Generation

Zhichao Zhang, Ming Xu, Jinsheng Deng, Hui Chen
article en

Abstract

Abstract Text-to-4D scene generation is currently impeded by three primary challenges: the prohibitive memory cost of training high-dimensional diffusion models, the semantic misalignment arising from the failure to disentangle static object identity from dynamic motion in text conditioning, and the scarcity of high-quality datasets that accurately correlate subjects, motions, and environments. To surmount these challenges, we present Swin4D , an end-to-end framework for efficient and high-fidelity text-to-4D synthesis. First, we introduce a Block-wise Variable Conditional Diffusion strategy, which mitigates memory constraints and prevents overfitting by dynamically reconfiguring spatial dependencies during training. Second, we propose Global–Local Alignment , a hierarchical guidance mechanism that explicitly disentangles global scene context from local subject-action semantics via separate T5 and CLIP encoders, ensuring spatiotemporal consistency and fine-grained semantic fidelity regardless of prompt length. Finally, we address data limitations by curating RealSynth4D , a large-scale hybrid dataset combining 108k real-world captures with 72k AI-synthesised scenes, establishing robust correlations between subjects, actions, and complex backgrounds to significantly enhance model generalisation.

International Journal of Computer VisionVol. 134(12)
National University of Defense Technology (CN), PLA Academy of Military Science (CN)
National Natural Science Foundation of China, Science and Technology Innovative Research Team in Higher Educational Institutions of Hunan Province
Generative Adversarial Networks and Image Synthesis
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon