acceptodds
Withdrawn from ICLR 2027

Response-Disagreement Preconditioning for Causal Video Distillation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Distilling a bidirectional video diffusion model into a few-step causal generator requires transferring text-conditioned behavior across fundamentally different temporal factorizations. The teacher denoises with full-sequence context, whereas the student makes sequential, prefix-conditioned predictions. In asymmetric distribution matching distillation (DMD), the causal student is updated using the discrepancy between an online estimate of its rollout score and a classifier-free-guided teacher score. This discrepancy provides the base correction for distribution matching, but does not explicitly characterize that correction relative to differences in how the teacher and student score fields respond to text conditioning. We introduce Null-Relative Response-Preconditioned DMD (NRP-DMD) to exploit this response disagreement within the existing DMD update. At the same noised student rollout and diffusion timestep, we evaluate both score models under the target text and a null condition. For each model, the null-to-text change defines a text-induced score response, and the disagreement between the teacher and student responses defines a sample-dependent axis in score space. NRP-DMD uses this axis to precondition the existing guided DMD update, increasing the gain of its response-aligned component while leaving the orthogonal component unchanged. NRP-DMD modifies only the distillation procedure and requires neither architectural changes nor additional inference-time computation. Across two causal distillation baselines, NRP-DMD consistently improves generation quality over both 5- and 30-second horizons while maintaining strong text alignment and temporal coherence.

Primary area:
generative models
Keywords:
video generation, diffusion models, diffusion distillation, distribution matching distillation, causal video generation

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_iLmROpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes NRP-DMD for distilling bidirectional video diffusion models into few-step causal generators. Motivated by the different temporal factorizations of the teacher and student, it uses their disagreement in text-conditioned score responses as additional directional information for distillation. Specifically, NRP-DMD computes the teacher–student disagreement between null-to-text score responses and uses it to precondition the original DMD update, selectively amplifying the aligned component while preserving the orthogonal component. The method requires no architectural changes or additional inference-time computation. Experiments on Self Forcing and Causal Forcing show consistent improvements in VBench Total and Quality for both 5-second and 30-second generation, supported by extensive ablations and diagnostic analyses.

Critical strengths
  1. Simple and practical design with no inference overhead. NRP-DMD reweights the existing DMD update rather than introducing a new objective or modifying the generator architecture. It can be directly integrated into Self Forcing and Causal Forcing, with only about 10% additional training time and no extra inference cost.
  2. Strong controlled evidence for the proposed response-disagreement direction. Uniformly amplifying the DMD signal, shuffling the response direction across samples, or using alternative axes such as the teacher response alone cannot reproduce the gains. These controls support that the improvement comes specifically from the sample-dependent teacher–student response disagreement.
  3. Consistent improvements across different baselines and generation horizons. NRP-DMD improves VBench Total and Quality on both Self Forcing and Causal Forcing for 5-second and 30-second generation. The 5-second improvements in Total and Quality are further supported by paired bootstrap confidence intervals, strengthening the robustness of the empirical results.
Critical weaknesses
  1. The temporal-factorization motivation is not directly isolated. The paper argues that response disagreement arises from the mismatch between bidirectional and causal temporal modeling, but the experiments mainly show that the disagreement direction is useful. A causality-matched teacher–student control would better validate this central interpretation.
  2. The performance gains are consistent but relatively modest. On 5-second generation, the improvements in VBench Total and Quality are small, and gains in Semantic are inconsistent. Although bootstrap results support the Total/Quality improvements, the practical magnitude remains limited.
  3. The empirical validation is still somewhat narrow. The method is evaluated only on Wan2.1-based Self Forcing and Causal Forcing, with generation horizons up to 30 seconds. Broader validation across different backbones or causal distillation frameworks would strengthen the claim of general applicability.
Questions for authors
  1. Can the authors provide a causality-matched teacher–student control, or another experiment that isolates the effect of temporal-factorization mismatch? This would help determine whether the benefit of response disagreement is specific to bidirectional-to-causal distillation or reflects a more general property of conditional DMD.

  2. Given the relatively modest gains in the 5-second setting and the inconsistent changes in Semantic scores, can the authors report paired bootstrap confidence intervals for the 30-second results, similar to those provided for the 5-second evaluation, and, if available, results across independent training seeds? This would help assess the robustness of the reported improvements.

  3. Do the authors have results on an additional backbone or causal distillation framework beyond the current Wan2.1-based Self Forcing and Causal Forcing settings? Such evidence would materially strengthen the claim that NRP-DMD generalizes beyond the specific experimental setup used in the paper.

Reviewer_4zLWOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper targets asymmetric distribution matching distillation (DMD) of a bidirectional video diffusion teacher into a few-step causal student. In addition to the standard target-conditioned fake and CFG-guided real scores, it evaluates both the online fake-score model and the frozen teacher under a null text condition at the same noised student rollout and timestep. The difference between the two models' null-to-text score responses, dt=(sfakec−sfake∅)−(srealc−sreal∅)d_t = (s^{c}_{\text{fake}}-s^{\varnothing}_{\text{fake}}) - (s^{c}_{\text{real}}-s^{\varnothing}_{\text{real}}), defines a sample- and block-dependent axis. The DMD discrepancy Δst\Delta s_t is then preconditioned by Pt=I+λ dtdt⊤/∥dt∥2P_t = I + \lambda\, d_t d_t^\top/\|d_t\|^2, i.e.\ its component along dtd_t is scaled by 1+λ1+\lambda while the orthogonal component is unchanged (``NRP-DMD''). The fake model's null branch is obtained by condition dropout (ρ=0.1\rho=0.1); λ=0.2\lambda=0.2. The method is applied to Self Forcing and Causal Forcing (Wan2.1-1.3B student, 14B teacher). On 5-second VBench, Total improves by +0.41+0.41 (Self Forcing) and +0.35+0.35 (Causal Forcing); on 30-second VBench-Long by +0.27+0.27 and +1.06+1.06. Ablations compare direct addition of dtd_t, a norm-matched uniform gain, a norm- and alignment-matched shuffled direction, alternative preconditioning axes, block placement, and λ,ρ\lambda,\rho sweeps; prompt-level paired bootstrap CIs are given for the 5-second results. Training overhead is about 10%; inference is unchanged.

Critical strengths

1. Careful controls. The Uniform Gain and Shuffled Direction controls are matched in signal norm and in alignment with the base DMD signal, Direct Addition isolates the projection, Appendix C.3 tests two alternative axes (teacher response alone, target-conditioned residual), and Appendix E shows that suppressing the aligned component (λ=−0.2\lambda=-0.2) hurts. Paired prompt-level bootstrap CIs are reported. This is the main reason my assessment is not more negative: the sign of the effect is consistently reproduced under the stated protocol.

2. Practicality. The change is training-only, adds two extra score evaluations per update (about 10% wall-clock), needs no architectural change and no inference cost, and plugs into two different forcing frameworks. If the effect is real, it is essentially free.

Critical weaknesses

1. The "Quality" gain is small, and the per-dimension data show it is mostly an increase in motion. The headline gains are +0.35 to +0.41 VBench Total with bootstrap CI lower bounds of about 0.1. The paper's own numbers show that perturbations of comparable size arise from incidental changes: turning on condition dropout alone moves Total by -0.39 and Semantic by -0.96 (Table 3), so sub-point differences on VBench should be interpreted with care. More importantly, Table 5 shows that the 5-second Quality gain on Causal Forcing is almost entirely Dynamic Degree (65.8 to 73.9), while Background Consistency, Temporal Flickering and Aesthetic Quality all decrease; at 30 seconds Dynamic Degree again jumps (60.6 to 70.5) and Imaging/Aesthetic Quality slightly drop. The abstract's "consistently improves generation quality ... while maintaining ... temporal coherence" is not what the per-dimension data show; a more accurate description is that NRP-DMD produces more motion, which VBench's Quality aggregate rewards. Whether that is a quality improvement or a motion-vs-consistency trade needs a human study or a motion-matched comparison. I also note the official Causal Forcing checkpoint has Dynamic Degree 83.0 at 30 s versus 60.6 for the authors' reproduction, i.e. the baseline differs from the released model along exactly the dimension that drives the reported gain.

2. No logical link to the causal setting. The method is motivated by the bidirectional-vs-causal mismatch, but nothing in the construction refers to prefix conditioning, the KV cache, or the temporal factorization: the null condition is defined identically for teacher and student, and the preconditioner applies verbatim to any DMD setup. The factorization mismatch is equally present in the unconditional residual et(∅)e_t(\varnothing), and no argument is given for why the text-response part should be specific to causal distillation. The choice to amplify rather than suppress, and the gain λ\lambda, are admitted to be empirical (App. C.2). A single experiment on a non-causal DMD student would show whether this is a general DMD trick or something specific to bidirectional-to-causal transfer.

3. The projection is direct addition with a sample-dependent coefficient, so the Projection-vs-Addition gap is unexplained. Since Δst=dt+et(∅)−γgt\Delta s_t = d_t + e_t(\varnothing) - \gamma g_t (with gt=srealc−srealc−g_t = s_{real}^{c}-s_{real}^{c^-}), Eq. (10) gives PtΔst=Δst+λctdtP_t\Delta s_t = \Delta s_t + \lambda c_t d_t with ct=1+⟨et(∅)−γgt,dt⟩/∥dt∥2c_t = 1 + \langle e_t(\varnothing)-\gamma g_t, d_t\rangle/\lVert d_t\rVert^2. NRP-DMD is therefore Direct Addition scaled by a scalar ctc_t that can be negative. "Preserving the zeros of Δst\Delta s_t" is a measure-zero property and cannot explain a 1.1-point gap (+0.35 vs -0.78 Total). Either ctc_t matters a great deal, which needs to be shown, or much of the gap is noise. Relatedly, Table 7 compares the alternative axes at ρ=0\rho=0 with NRP-DMD at ρ=0.1\rho=0.1; at equal ρ=0\rho=0, Teacher Response (84.60 / 85.52) and Response Disagreement (84.69 / 85.78) are indistinguishable, so the "disagreement" ingredient is not shown to matter.

Questions for authors

Q1. Please provide the per-dimension VBench breakdown for the Self Forcing pair and for both 30-second pairs, plus either a small paired human preference study (visual quality, temporal consistency, motion) or a comparison in which Dynamic Degree is matched, so the Quality gain can be separated from the increase in motion.

Q2. Why does the reproduced Causal Forcing baseline differ so strongly from the official checkpoint at 30 seconds (Dynamic Degree 60.6 vs 83.0, Temporal Flickering 96.0 vs 94.1)? Which should be regarded as the reference?

Q3. Does NRP-DMD help a bidirectional (non-causal) DMD student distilled from the same teacher? A yes or no is informative either way.

Q4. Please report the distribution of the scalar ct=⟨Δst,dt⟩/∥dt∥2c_t = \langle \Delta s_t, d_t\rangle/\lVert d_t\rVert^2 (equivalently, of the coefficient multiplying λdt\lambda d_t in the NRP-DMD update) over samples, blocks and timesteps: what fraction is negative, and how does a variant that clips ctc_t to be non-negative, or fixes ct=1c_t=1 (Direct Addition), behave? This is needed to understand why Projection and Direct Addition differ by 1.1 points when they differ only by this coefficient.

Q5. Please compare the preconditioning axes at matched ρ\rho: Teacher Response and Target-conditioned Residual with ρ=0.1\rho=0.1 (and, if possible, a run using only the teacher response direction rrealcr_{real}^{c}, which needs no null branch of the fake model at all). If the teacher response alone matches NRP-DMD, the contribution should be reframed accordingly.

Q6. How many generator updates does the refinement stage use, and how well trained is the fake model's null branch under ρ=0.1\rho=0.1? A diagnostic (null-branch denoising loss vs a separately trained unconditional fake model, or sensitivity of dtd_t to the null-branch checkpoint) would support the claim that sfake∅s_{fake}^{\varnothing} estimates the prompt-marginal score.

Reviewer_4tMYOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposed NRP-DMD, which compute the disagreement between the teacher’s and the student’s to the prompt and unconditioned to align the DMD correction during training. The method compute the disagreement direction between teacher and student first by the non conditional responses from the prompt and second to the conditional one. From the VBench dataset Quality score improves while not reaching an improvement with Semantic score.

Critical strengths

⁠Focused intervention: the paper modifies the DMD update without changing the student architecture or inference procedure.
⁠Evaluation across two baselines and horizons: results on both Self Forcing and Causal Forcing provide evidence that the method is not limited to one setup.

Critical weaknesses

Unfortunately this paper is very far from my domain of expertise, as I already communicated the AC. Therefore, I will read the interaction of the authors with the other reviewers to clarify my doubts. Below I report the list of weaknesses I identified. Should the authors bring enough evidence in support of their results, I will consider changing my score.

⁠Semantic results are mixed. Semantic scores decline in some comparisons, including the 30-second Self Forcing result. I believe the claims about preserving text alignment should reflect this variation.
⁠External comparisons are difficult to interpret. The experiments reported in the paper combine models at different resolutions and possibly different frame rates and sampling protocols. It also needs comparison, or a clear scope limitation, relative to recent causal-distillation methods.
The method’s exact effect needs clear specification. The excerpts describe a rank-one pre-conditioner, but assessing reproducibility requires the precise normalization of the response direction, the role of its magnitude, and sensitivity to λ\lambda. Training-time overhead from null condition evaluations should also be clearly reported.

Questions for authors

Could the authors clarify how many random seeds were used, and can the authors report confidence intervals for the VBench differences? When d is small, its direction in more sensitive to noise and score estimation. The paper does not specificy a threshold or a stabilization strategy: could the authors comment on this point?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.