- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper changes how the DMD query is formed in Self-Forcing-style AR video distillation. Instead of re-noising the whole rollout at one timestep, it samples a per-chunk timestep vector and sorts it along generation order, so earlier chunks get less noise than later ones; the teacher’s timestep conditioning is fine-tuned to accept per-chunk timesteps. Separately, the fake score shares the generator’s backbone, with a small set of role-conditioning parameters telling the network which job it is doing. Empirically, the ordered query gives small but consistent VBench gains over several matched-noise controls, the final system edges out the locally evaluated Self Forcing checkpoint, and the shared fake beats an independent one at a lower fake-update ratio with less memory.
Critical strengths
- The experiments are thorough and the ablations are carefully controlled. Reversed landing below scalar is a particularly useful result, it rules out heterogeneous noise by itself as the explanation.
- The claims are stated carefully. The paper is candid about what it does not establish, distinguishes literature-reported from locally evaluated numbers.
- The paper is well writen.
Critical weaknesses
- The sharing gain is confounded. The fake loss also updates the generator, and the trunk gets 600 extra fake-objective updates, so the +1.3 at r=2 could be tracking, regularization, or just more updates. No shared r=5 row, and the independent causal r=5 it is compared to is a single run.
- The headline comparisons are thin: one checkpoint per policy in Table 3, one video per prompt in Table 5, Causal Forcing is cited but not compared, and there are no external 60-second baselines.
Questions for authors
- Can you add the shared r=5 row?
- Teacher adaptation details (batch size, lr, compute), and is the adapted teacher’s denoising loss similar across the four profiles?
- For the independent fakes, is the fake update done on a fresh post-step rollout like the shared model? And how was the AR Cosmos initialization obtained?
3 comments
I thank the author for their response. The extra experiments and added details have basically addressed my concerns. Although an external 60s baseline comparison was not added, this does not affect the roel of ordered queries within the ablation. Therefore, I'm inclined to maintain my current rating.
Thank you for following up. We’re glad the additional experiments and clarifications helped address your concerns.
Response to Reviewer F93Y
Thank you for highlighting the reversed-query control and for suggesting the shared r = 5 comparison.
(W1/Q1) Shared training at r = 5
As you point out, at r = 2 the shared generator backbone receives 600 updates from the fake loss in addition to its 300 generator updates. We therefore interpret the 1.30-point difference at this ratio as a comparison of the two training designs as a whole. Following your suggestion, we trained the shared model at r = 5 three times and added two runs of the independent causal baseline at the same ratio under the Table 2 protocol.
We will add these results to Table 2. Each entry below is the mean VBench Total ± sample SD over three training runs.
Raising r from 2 to 5 increased mean VBench Total by 0.77 points for the independent model and 0.02 for the shared model, narrowing the shared model's lead to 0.55 points. In this setting, the shared model reached nearly the same mean score with 600 fake updates as with 1,500.
(W2) Five-second and 60-second comparisons
Direct five-second comparison with Causal Forcing. Table 3 originally quoted the published Causal Forcing score of 84.04 VBench Total. We have now evaluated its released chunk-wise checkpoint using the same five-second VBench protocol as ViDA. Causal Forcing scored 84.37 and ViDA scored 85.62, a 1.25-point gain for ViDA. We will report this local evaluation alongside the published result in Table 3.
How many training runs support the ordering gain? In Table 3, we trained one model with ordered queries and one with matched scalar queries, both using the 14B teacher. The ordered model scored 1.12 VBench Total points higher in that pair. In a separate experiment with a 1.3B teacher, we kept the same shared causal design and trained each policy with three matched seeds. Ordered queries scored higher in all three pairs, by 0.99 points on average (Appendix Tables 7–8). We will state the run count alongside the Table 3 comparison.
What did we compare at 60 seconds? Table 5 compares the ordered and matched-scalar checkpoints from Table 3 to test whether the ordering gain persists in longer videos. It includes no external 60-second baseline. We generated one 60-second video per checkpoint for each of 128 MovieGen prompts, matching the rendering seed and extension settings within each pair. Across these 128 pairs, the ordered checkpoint scored higher on average on all six reported dimensions, including aesthetic quality (+1.61 points) and imaging quality (+1.35 points).
(Q2) Teacher adaptation cost and denoising loss across profiles
Adaptation settings and cost. For each Wan teacher, we trained only the timestep-conditioning parameters for 2,000 steps on 64 H100 GPUs, using a global batch size of 64 and AdamW with a learning rate of 2 × 10⁻⁶. We used video–text pairs from OpenVidHD-0.4M (Nan et al., 2025) and sampled the four noise profiles uniformly. Adapting each teacher once cost approximately 105 H100 GPU-hours for the 1.3B model and 545 for the 14B model. We then froze each checkpoint and reused it across student runs. Table 2 reports student post-training costs separately, with the same frozen 1.3B teacher for all configurations.
Denoising loss. During adaptation of the Wan 1.3B teacher used in Tables 1 and 2, the four profiles had similar mean flow-matching losses. The highest mean was 2.7% above the lowest over the full 2,000-step run and 3.4% above the lowest over the final 500 steps.
We will report the adaptation settings, costs, and per-profile losses in Appendix A.12.
(Q3) Independent fake updates and Cosmos initialization
Do the independent fake estimators also use fresh rollouts? Yes. When both the generator and the fake estimator are updated in the same iteration, we update the generator first. We then generate a fresh rollout with its updated weights, detach it, and use it for the fake update. On fake-only iterations, we also draw a fresh rollout from the current generator. We will clarify this order in Appendix A.9.
How was the Cosmos AR checkpoint obtained? We built it ourselves from the pretrained bidirectional Cosmos-Predict2.5-2B model. We converted its attention to chunk-causal attention and used ODE initialization to obtain a few-step autoregressive generator. Both the matched-scalar and ordered-query models in Table 4 started DMD post-training from this same checkpoint. We will add this detail to Appendix A.8.