- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes Rollout-Marginal Distillation (RMD), which scores generated chunks independently and subsequently applies video-level distillation to improve temporal coherence. It addresses error accumulation in long-horizon autoregressive video generation, where joint video supervision can reinforce artifacts in imperfect generated context. Trained on five-second rollouts, RMD outperforms Self Forcing and Causal Forcing on 60-second VBench-Long evaluations, with an additional qualitative example extending to 500 seconds.
Critical strengths
-
A clear distinction between generation context and scoring context. Equations (4)–(5) explicitly formulate the rollout-induced chunk marginal while preserving conditional autoregressive generation. This provides a concrete approach to separating local visual-quality supervision from the influence of imperfect generated context. Subsequent video-level refinement addresses the temporal relationships omitted by independent scoring (Sections 3.2–3.4).
-
Strong long-horizon results supported by relevant ablations. Evaluation covers 944 prompts with five generations per prompt. With chunk size one, the 60-second Total score exceeds both evaluated baselines by more than ten points; improvements also hold with chunk size three. Marginal-only, video-DMD-only, and random-exit comparisons, together with temporal-consistency metrics, support the complementary roles of independent quality supervision and temporal refinement. Inference retains the same architecture and sliding-window protocol without additional modules (Section 4, Tables 1–2).
Critical weaknesses
-
Incomplete training details limit attribution of the improvements. The video-DMD-only control is not explicitly related to full RMD in terms of total generator updates, fake-score updates, learning-rate schedules, and computational budget. The purpose and data source of continuation-teacher adaptation are explained, but adaptation sample counts, LoRA rank, and training schedules remain unspecified. These details are necessary to assess the contribution of the objective change separately from other training choices and to estimate reproduction cost (Sections 3.3–4.1, Table 2).
-
The explanation for artifact accumulation remains supported primarily by illustrative and indirect evidence. Equation (3) establishes that joint scoring depends on the surrounding sequence, and Figure 2 presents an example consistent with the proposed mechanism. However, these observations do not directly isolate how historical artifacts alter the teacher’s correction direction or establish how much this mechanism accounts for the long-horizon gains. A controlled analysis of scoring context, or a more explicit qualification of the causal interpretation, would strengthen the connection between the motivation and the experimental findings (Section 3.1, Figure 2).
Questions for authors
-
Does video-DMD-only match full RMD in generator updates, fake-score updates, and learning-rate schedules? Please specify the corresponding configurations.
-
What sample count, LoRA rank, training schedule, and additional computational cost are used for continuation-teacher adaptation?
-
With the generated chunk and scoring noise fixed, how does changing artifacts in the scoring context affect the teacher’s correction direction? What evidence distinguishes this mechanism from other possible explanations for the observed improvements?
Thank you for the constructive review and for recognizing our clear distinction between generation and scoring contexts, as well as the strength of our long-horizon results. The additional experimental settings you suggested help further substantiate our claims. We address your concerns below.
W1 & Q1: Budget-Matched Ablations for the Training Objective
Our original
video-DMD-onlyablation was designed to measure the impact of omitting the initial rollout-marginal training phase, which inherently resulted in fewer total updates. To isolate the contribution of the training objective under equal compute, we added two budget-matched ablations evaluated on the 60-second VBench-Long protocol:These ablations confirm that RMD's performance gains do not simply come from extended training compute. Extending video-DMD at its original learning rate provides only a marginal gain (Total: 74.08). Furthermore, forcing video-DMD into our sequential learning rate schedule destabilizes the model (Total collapses to 62.17). This reinforces our original conclusion: the rollout-marginal objective itself is the primary driver of the long-horizon improvements.
W1 & Q2: Continuation-Teacher Adaptation Details and Cost
To detail the adaptation process, the continuation-teacher and fake-score models are one-time offline initializations.
Because this is purely an offline pre-training setup to prepare the scoring targets, student generation during inference remains standard, incurring zero additional modules, memory, or computational cost.
W2 & Q3: Controlled Analysis of the Scoring Context
We agree that a controlled analysis of the scoring context strengthens the connection between our motivation and experimental findings. Following your suggestion to fix the generated chunk and scoring noise, we conducted this targeted evaluation.
For each of six prompts, we extracted a 9-latent-frame window. Across all conditions, the 3-frame target chunk, diffusion timestep, and scoring noise were kept identical; only the 6-frame history shown to the scoring models was modified. We introduced two types of history interventions:
We then measured the relative change and directional shift (cosine similarity) of the diagnostic DMD correction signal, g=x0fake−x0real:
These results verify our theoretical mechanism. Progressively degrading the generated history substantially changes the score-difference signal, even when the target chunk and scoring noise are held identical.