- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper proposes OPF: On-Policy Forcing, a fully autoregressive training-to-distillation pipeline for few-step video generation. Following Causal Forcing, it first trains an AR diffusion teacher with teacher forcing and initializes a few-step student using causal ODE distillation. Its main modification is to use AR models for both real- and fake-score estimation during on-policy DMD, conditioning both on clean student-generated prefixes. Experiments on Wan2.1-T2V-1.3B compare four AR/BI score configurations and report improvements in VBench quality and motion, supported by repeated training runs and paired bootstrap intervals.
Critical strengths
-
Conditioning the teacher on a clean generated prefix and the current noisy block is a reasonable design choice. This matches the conditional denoising problem encountered during autoregressive generation and enables the same AR teacher to support both initialization and subsequent distillation.
-
The empirical investigation is relatively thorough within the evaluated setting. The paper includes four real/fake score configurations, a fake-score initialization control, three-seed comparisons, training trajectories, and prompt-level bootstrap intervals. These provide useful evidence that the proposed recipe can improve quality and training stability on the tested backbone.
Critical weaknesses
-
The experiments do not sufficiently distinguish the benefits of clean-prefix conditioning from those of removing future context. Moving from BI/BI to AR/AR changes both simultaneously: the AR score models observe clean preceding blocks and cannot attend to future blocks, whereas the BI models observe the full noisy video. Consequently, the improvement may arise from more informative, inference-aligned prefix conditioning, from excluding future information, or from their combination. Since the paper attributes its gains to eliminating a conditioning mismatch, separating these effects is important to establishing its central contribution.
-
The argument against bidirectional supervision does not adequately distinguish conditional matching from joint distribution matching. An AR teacher naturally estimates the score of the current noisy block conditioned on a clean prefix. However, a matched BI/BI pair can estimate scores of the full noised video distributions, and DMD does not generally require score estimators to share the generator's causal architecture. The collapse of AR/BI therefore demonstrates a problem with mixing differently conditioned scores, but does not by itself establish that BI/BI supervision is inappropriate for an AR generator. The paper should clarify the objective induced by its conditional scores and justify the broader interpretation of its ablations.
Questions for authors
-
Can the authors separate the effect of clean-prefix conditioning from that of excluding future context? Controlled comparisons that vary prefix noise and future visibility independently, using score models trained for the corresponding conditioning settings, would help establish which factor accounts for the gains.
-
What precise objective does Equation (7) optimize: full-video joint distribution matching or prefix-conditioned blockwise matching? How are gradients through generated prefixes handled? Why should the failure of mixed AR/BI scores imply a limitation of a matched BI/BI pair?
The proposed recipe appears promising, but its central attribution remains unclear. If the authors convincingly disentangle clean-prefix conditioning from future-context masking and align their claims with the resulting evidence, I'm willing to raise my score.
5 comments
Conditional and joint score targets: interpretation of Equation (7) (2/3)
W2/Q2: what objective does Equation (7) represent? Our update uses clean-prefix conditional score matching, rather than the exact gradient of a full-video joint KL. A matched BI/BI pair is also a valid joint-distribution-matching design for an AR generator. The generator's causal factorization does not require its supervising scores to have the same conditioning structure.
Conditional versus joint scores. Let denote the student's generated clean history. The ideal real-score targets are
The AR target perturbs the current block while conditioning on clean generated history. The BI target uses a jointly perturbed video, including noisy history and future blocks. The fake-score targets follow the corresponding conditioning.
For a fixed jointly noised distribution, causal factorization gives
This establishes two differences. First, the joint score includes derivatives of later conditional factors with respect to the current block. Second, even its first term conditions on noisy history rather than OPF's clean history. Thus, removing future terms from a jointly noised score does not recover OPF's clean-prefix conditional score.
Ideal conditional objective. For a fixed prompt and fixed history distributions , define the conditional KL for block as
The reference objective is
Here , and follows the training timestep distribution. The history distribution is held fixed during differentiation. RE denotes the distribution induced by random-exit sampling and subsequent perturbation. With exact unguided conditional scores, appropriate time weights, and differentiation through the corresponding conditional sampler, the score difference admits this conditional-KL interpretation.
In practice, histories are generated by the current student but detached for the generator update. The distinction matters because the KL chain rule is an identity of objective values, not permission to ignore the parameter dependence of generated histories when differentiating a full-video KL.
Moreover, the implementation includes real-score guidance, sample-dependent normalization, and truncated generator gradients. We therefore use the conditional KL as a reference interpretation, not as a claim that the implemented update is its exact full gradient.
Relation to BI/BI. Both matched score pairs provide valid distribution-matching signals, but for different corruption and conditioning schemes. The failure of mixed real/fake conditioning does not invalidate matched BI/BI. Our contribution is an empirical comparison of these designs under controlled training protocols, together with the conditioning-factor controls reported in Part 1.
Implemented surrogate, random-exit training, and gradient paths (3/3)
W2/Q2: what is actually differentiated? We clarify the implemented surrogate and its gradient paths to distinguish the practical update from the ideal conditional objective in Part 2.
Random-exit training. At each iteration, an exit step is sampled uniformly from the four denoising steps, shared across blocks and synchronized across ranks. Each block stops at and supplies its clean prediction as the training sample. Thus, training uses shared-random-exit rollouts rather than always generating the final four-step output and differentiating only an intermediate pass.
Implemented surrogate. Let concatenate the generated clean block predictions, denote the fake clean-sample prediction, and the guided real prediction. The real prediction uses negative-prompt guidance,
while the fake prediction uses the conditional prompt without guidance.
For each video, the implementation computes
\ell_{\mathrm{sur}}=\frac{1}{2D}\left\\|\tilde{x}_0-\operatorname{sg}(\tilde{x}_0-d)\right\\|_2^2,
where counts latent elements and denotes stop-gradient. The loss is averaged across the batch.
Because the regression target is detached, the computed per-video generator gradient is
where contains only the generator paths retained during backpropagation.
| Gradient path | Implementation |
|---|---|
| Selected denoising pass in each block | Gradient retained |
| Earlier denoising passes | No gradient |
| Generated-prefix KV-cache writes | No gradient |
| Score predictions, prefixes, and normalizer | Detached / evaluated without gradients |
| Rollout used to train the fake score | Detached from generator |
Generated history remains available to later blocks during forward computation. Truncation removes backward paths through the generation and caching of that history; each block still receives its own direct gradient through the selected denoising pass.
Why truncation does not make AR/AR and BI/BI equivalent. Two distinct dependencies must be separated. The BI joint score can depend on future noisy blocks through the distribution's score function. Generator-gradient truncation instead removes particular paths through the generator Jacobian. Detaching generated histories does not remove future-dependent terms already present in a BI score.
Likewise, evaluating the real and fake scores without gradients does not discard their supervision: their numerical predictions form , which is applied through .
The per-video normalizer also couples update magnitudes across blocks, even though each AR score observes only its permitted context. Hence our implementation should not be described as independent optimization of every block or as the exact gradient of a full-video KL.
Summary. Equation (7) is implemented as a guided, normalized, gradient-truncated score-difference surrogate on student-generated histories. The conditional-KL objective explains the intended score target under idealized assumptions; the expressions above specify the actual update. Together with the factor-separation controls, this clarifies both what distinguishes AR/AR from BI/BI and what the implementation optimizes.
Conditioning controls: separating clean-prefix and future-mask effects (1/3)
We thank the reviewer for the focused questions about the conditioning factors and the objective optimized by our method. We have completed the additional conditioning control and clarify the score targets and implemented gradients below.
W1/Q1: clean-prefix conditioning versus future visibility. The original AR/AR–BI/BI comparison changes both factors. We introduce an intermediate causal-noisy control, which observes noisy history but masks future blocks. Both the real and fake scores in each pair use the corresponding conditioning:
AR/AR versus causal-noisy tests clean versus noisy history with future blocks masked. Causal-noisy versus BI/BI tests future visibility while retaining noisy history. Each score model is trained for its own conditioning, rather than changing the attention mask of a pretrained BI model at inference time.
Matched protocol and results. All three teachers use the same OpenVid data, matched effective sample exposure, optimizer and learning-rate settings, and 1,200 teacher updates. Both noisy-history configurations use a shared noise level across blocks. Each fake score starts from its corresponding real-score checkpoint, and all students share the same causal ODE initialization and downstream DMD protocol. Each setting uses one DMD training seed.
What the controls show. With future blocks masked, clean-prefix conditioning improves Total over noisy-prefix conditioning by +0.19/+0.09/+0.18 at 120/240/360 generator updates. The positive difference appears at all three checkpoints.
With history kept noisy, future masking changes Total by −0.06/+0.16/+0.14 at the same checkpoints. Thus, masking future blocks gives higher Total at 240 and 360 updates, though not at 120. At 360 updates, the AR/AR–BI/BI gap of +0.32 separates into +0.18 for the clean-prefix contrast and +0.14 for the future-masking contrast along this three-configuration comparison.
These results provide more specific evidence than the original endpoint comparison: clean-prefix conditioning shows a positive difference throughout the evaluated trajectory, while future masking shows a positive difference at the later checkpoints. The effects are modest, and each setting uses one DMD seed.
The controls are conditional comparisons under matched training exposure, not a complete 2×2 factorial experiment. They do not estimate the interaction involving clean history with future visibility. We therefore attribute the observed differences to the tested conditioning configurations, rather than claiming that causal masking alone explains the full AR/AR advantage.