Back to Pixels: Lifting Latent Video Diffusion into Pixel Space
Frontier video generation models work in a VAE latent space that compresses video in both space and time. This compression made training tractable, but now limits high-resolution, controllable, and interactive generation -- doubling resolution quadruples token count, conditions apply to groups of frames rather than individual ones, and each response must wait for the rest of its multi-frame block. We present PixVid, a simple yet effective method that lifts a pretrained latent video model to directly generate every frame in pixel space. PixVid keeps the diffusion transformer and replaces the VAE decoder with a convolutional head that renders its outputs into pixels. We find that predicting clean video rather than velocity, together with a larger noise shift, is key to quality. Finetuned from Wan2.1-14B, PixVid is competitive with the latent baseline within $\sim 1.3$M training clips. Working framewise in pixels, it generates at $2\times$ higher resolution with the same per-frame token count, keeps depth-conditioned video editing aligned under fast motion, and responds to each action at the next frame. Project page: https://chenshuo20.github.io/PixVid/
Authors
- Chris Rockwell (ORCID: https://orcid.org/0000-0003-3510-5382)
- Tianpei Gu (ORCID: https://orcid.org/0000-0003-3173-8895)
- Desai Xie
- Aryan V S
- Yanke Song
- Shuo Chen
- Ming-Hsuan Yang
- Dennis Yuren Menn
- Yong-Hyun Park
- Nan Liu
- Guande He
- Xun Huang
Publication Details
- Journal
- arXiv (Cornell University)
- Published
- 2026-11-12
- DOI
- https://doi.org/10.48550/arxiv.2611.17336
- Primary Topic
- Generative Adversarial Networks and Image Synthesis
- Type
- preprint