Generative models for automated visual effects in film post-production

To address the core bottlenecks of high cost, long production cycle, and high technical barriers in traditional film post-production visual effects (VFX), this paper proposes CineFX-Diff, a cascaded conditional diffusion model for automated visual effects generation. The proposed framework comprises four tightly coupled functional modules—multi-modal condition encoding, cascaded diffusion backbone, temporal consistency constraint, and physics-aware refinement—which jointly enable multi-modal controllable generation, inter-frame temporal coherence, and physics soft-prior injection within a unified architecture, taking text prompts, reference frames, and spatial masks as inputs to produce high-fidelity VFX videos through a unified end-to-end inference pipeline. Based on a self-collected and annotated film VFX dataset comprising 6800 high-quality clips spanning six categories (explosion, fire/smoke, water/liquid, lightning, magical FX, and destruction), CineFX-Diff achieves superior performance compared to representative state-of-the-art methods on five core metrics—FVD, FID, LPIPS, CLIP-Sim, and temporal consistency—with a 19.3% reduction in FVD compared to the second-best baseline. Meanwhile, the model reduces the parameter count by 71.4% and 66.5% relative to Imagen Video and Make-A-Video, respectively, and by 27.0% relative to Stable Video Diffusion, while achieving 1.25 $$\times $$ -−2.38 $$\times $$ faster inference depending on the baseline, and obtains significantly higher subjective scores from 20 professional film post-production workers compared to baseline methods ( $$p<0.01$$ ). Ablation studies further verify the synergistic effectiveness of each module. The primary contribution of this work is the systematic engineering integration and domain-specific adaptation of established generative techniques for the VFX generation task, rather than a fundamentally new generative paradigm. This work provides a feasible technical pathway and a curated evaluation dataset for the large-scale application of generative artificial intelligence in film post-production.