JVT: Just Video Transformer makes pixel-space video diffusion simple
JVT generates video directly in pixel space, with quality the authors report is competitive with latent diffusion baselines.
The model uses a large-patch Transformer without a pretrained autoencoder, auxiliary detailers, cascaded stages or U-Net-style skip connections. The researchers examine patch embeddings, register tokens and adaptive layer normalization to balance quality and computational efficiency. They report competitive generation in both class-conditioned and unconditional settings. Joint RGB-disparity generation also improves camera following over RGB-only baselines and provides explicit scene geometry without an additional reconstruction model at inference.
HF Daily Papers' note
The model uses a large-patch Transformer without a pretrained autoencoder, auxiliary detailers, cascaded stages or U-Net-style skip connections. The researchers examine patch embeddings, register tokens and adaptive layer normalization to balance quality and computational efficiency. They report competitive generation in both class-conditioned and unconditional settings. Joint RGB-disparity generation also improves camera following over RGB-only baselines and provides explicit scene geometry without an additional reconstruction model at inference.
HF Daily Papers' note
score 5