- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper presents an 11B video diffusion model with a reported marginal training cost of approximately $200k. It combines FLUX initialization, progressive T2V/I2V training, and separate image/text guidance, and introduces a high-compression Video DC-AE variant. Its principal research claim is that these design choices yield transferable insights into compute-efficient video generation. Evidence includes VBench, human evaluation, and 22 exploratory training runs.
Critical strengths
-
The paper provides a practical training recipe with explicit stage-level compute accounting and useful distributed-training details, making it informative for researchers working with limited resources.
-
Video DC-AE reduces the DiT token count by approximately four times while retaining competitive reconstruction quality, offering a useful component for studying efficient video generation.
Critical weaknesses
-
Video DC-AE lacks quantitative validation in downstream generation. The main VBench and human evaluations use the HunyuanVideo VAE, while the Video DC-AE variant remains incompletely converged and produces blurrier outputs. Neither its generation-quality trade-off nor the claimed convergence benefit of DINOv2 latent distillation is quantified. Reconstruction and throughput results alone do not establish the effectiveness of this central component.
-
The experiments do not establish the claimed sources of training efficiency. The FLUX ablation removes inference-time image conditioning while retaining pretrained initialization. The 22-run analysis compares terminal training losses across different datasets and loss weightings, without a common evaluation objective or corresponding generation-quality measurements. These experiments cannot substantiate the claimed improvements in quality per unit compute.
-
The additional contribution beyond prior work is insufficiently established. High-dimensional latent optimization and foundation-model alignment were already studied in VA-VAE, while separate image and text guidance scales appear in InstructPix2Pix. Extending these ideas to video could be valuable, but the paper provides limited quantitative evidence of new findings specific to video. In particular, the proposed guidance schedule lacks a controlled comparison against static, separately tuned image and text guidance.
Questions for authors
-
What quantitative evidence establishes Video DC-AE’s generation quality and the benefits of DINOv2 distillation? Does dynamic guidance improve upon separately tuned, static image and text guidance?
-
Given that the FLUX ablation retains pretrained initialization and the 22-run analysis compares different datasets and loss objectives, what evidence isolates the claimed training-efficiency gains?
-
Can the authors reconcile the VBench total of 84.3 in Table 3 with 83.6 in Figure 8 and Table 11, specifying the checkpoints and evaluation settings used?
6 comments
W3: Insufficient novelty beyond prior work
We respectfully disagree that the contributions reduce to known techniques, though we acknowledge the paper could have been clearer about differentiating from prior work:
vs. VA-VAE (Yao & Wang, 2025). VA-VAE studies the reconstruction-vs-generation trade-off for image diffusion with standard-compression AEs (4×8×8 or similar). Our Finding (iii) is specific to video high-compression AEs (4×32×32) where the channel–convergence tension is significantly more acute — the 32× spatial compression means the latent space carries a much heavier information bottleneck, and the finding that 128 channels outperform 256 channels for generation despite worse reconstruction is a non-trivial extension to this high-compression regime. Finding (iv) — DINOv2 distillation for video AE latents — is entirely novel: VA-VAE proposes CLIP-based alignment for image AEs, while we show that DINOv2 intermediate features are effective for video AE latents in the high-compression regime.
vs. InstructPix2Pix (Brooks et al., 2023). InstructPix2Pix uses static, separately tuned image and text guidance scales for image editing. Our contribution is the dynamic, frame- and step-dependent schedule (Finding v) that addresses a problem unique to video generation: the need for stronger image guidance on frames temporally close to the conditioning image and at early denoising steps, combined with an oscillation trick to suppress flickering. As shown in New Table B, our dynamic schedule provides a measurable +0.4 VBench improvement over separately tuned static guidance, and the oscillation trick specifically addresses temporal flickering — a video-only concern absent in image editing.
We have expanded the Related Work section to make these distinctions clearer.
Questions for Authors
Q1 (DC-AE generation quality and DINOv2 distillation; dynamic vs. static guidance): Addressed above — see New Tables A and B, and New Figure B. Summary: DC-AE achieves 80.1 VBench Total (−3.5 vs. main model) for 10× speedup; DINOv2 distillation provides +1.7 VBench at matched compute; dynamic guidance provides +0.4 over separately tuned static guidance.
Q2 (Evidence isolating training-efficiency gains): Addressed above — clarified the scope of the FLUX ablation and 22-run analysis, and added VBench evaluations on representative runs.
Q3 (VBench 84.3 vs. 83.6): Addressed above — this was caused by using different checkpoints. We have re-run all evaluations from a single checkpoint and corrected the tables.
The additional experiments address DC-AE’s generation quality and include component ablations, but I still have three questions:
-
The FLUX ablation tests the role of image conditioning at inference while retaining pretrained initialization. It therefore does not support the claim in Section 7.1 that motion learning is independent of FLUX.
-
Tables 5 and 19 state that they use the same checkpoint, yet report Aesthetic/Imaging Quality scores of 62.4/67.8 and 65.1/72.4, respectively. Please clarify whether these differences reflect different evaluation settings or omissions when updating the tables.
-
The distinction from prior work needs to be explained more precisely. VA-VAE already studies DINOv2-based latent alignment. Beyond applying this approach to video, please clarify the specific methodological adaptations and new findings arising from the high-compression video setting.
We thank Reviewer xND3 for recognizing the practical training recipe with explicit compute accounting and the usefulness of Video DC-AE for efficient video generation research.
W1: Video DC-AE lacks quantitative downstream generation validation
We fully agree and have addressed this comprehensively. Please see:
Specifically, for the DINOv2 distillation benefit:
We acknowledge that the DC-AE variant remains incompletely converged and produces blurrier outputs. As stated in the paper's Limitations (Appendix T, L5) and Future Work (Section 8), closing this reconstruction gap is a key future direction. The main model uses HunyuanVideo VAE, and we position DC-AE as a research contribution and speed-critical deployment option, not a replacement.
W2: Experiments do not establish claimed sources of training efficiency
We appreciate this nuanced critique and address both sub-points:
FLUX ablation scope. The reviewer correctly notes that our FLUX ablation (Table 3) removes inference-time image conditioning while retaining pretrained initialization, so it isolates FLUX's contribution to visual quality via first-frame fidelity, not the entirety of cost savings from FLUX initialization. We have clarified this in the revision: Table 3 shows that FLUX contributes approximately 2.5 VBench points concentrated in Aesthetic Quality (−4.8) and Imaging Quality (−4.7), while motion dimensions remain essentially unchanged (Motion Smoothness −0.4, Temporal Flickering −0.3). This confirms that FLUX provides visual quality (the "image prior") while our $200k training provides motion — these are complementary, not substitutable contributions.
A clean ablation removing FLUX initialization entirely (training from scratch) would cost an additional >$100k and roughly double our total budget. We acknowledge this limitation explicitly in the revision while noting that the T2V-only mode (81.8 VBench Total) remains competitive with HunyuanVideo (82.2), a dedicated T2V model, suggesting our training recipe contributes meaningfully beyond the FLUX initialization.
22-run analysis scope. We agree with the reviewer that the 22-run analysis compares different datasets and loss objectives, which limits its ability to isolate individual efficiency factors. We clarify the analysis's intended contribution: it is a natural ablation (Section 7.2) — a pragmatic alternative to a prohibitively expensive controlled grid. Its value is threefold:
We have added a VBench evaluation on representative runs from the 22-run analysis to connect terminal loss to generation quality: