Regime-Aware Temporal Granularity for Causal Video World Models: When Does Adaptivity Pay?

Causal video generators produce a rollout one temporal chunk at a time, so spending expensive per-frame compute only on hard segments looks attractive. We show that this idea rests on three preconditions—cost, signal and endpoint—and that all three are regime properties rather than universal facts. On a matched two-GPU study (Tesla T4 and A100-80GB) of a causal diffusion backbone, we (i) measure the per-forward cost against the schedule break-even and find the same mode pair on opposite sides of the inequality; (ii) show that hand-crafted context statistics cannot predict coarse-unsafe segments (AUROC 0.491), that an intrinsic probe on the generator's own prefill activations reaches 0.798 on one GPU and inverts to 0.143 on the other, and that only a multi-feature model over the probe and a sampler convergence residual transfers across both (0.842 and 0.833); and (iii) show that an endpoint independent of the compared modes is mandatory, because the internal reference reverses between GPUs. We then derive a regime map that states exactly when adaptivity pays—only when the cheap mode is cheaper and worse for a stated objective. Across two GPUs and three independent endpoints—latent (both GPUs), decoded-pixel and temporal flicker (T4)—our operating points fall outside that region: the coarse mode was never worse in quality, so under the stated objective the optimal policy is a fixed mode, and a router is unnecessary. We also report a training-free seam repair that reduces the switch-boundary discontinuity by 25% while improving fidelity, quantifies drift propagation, and records three boundaries: no single signal transfers across GPUs, drift is linear and policy-insensitive, and a pixel-only decision-model critic is at chance. All results, figures and audited result files are public.