- Rating
- 3
- Confidence
- 3(+1)
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3(+1)
Read review
Paper summary
This paper proposes Adversarial Gradient Forcing (AGF) to address error accumulation in few-step autoregressive video generation. The core idea is to use Distribution Matching Distillation (DMD) gradients from future predictions to construct targeted perturbations to model-generated history, and jointly optimize predictions conditioned on unperturbed and perturbed histories. The authors also introduce prediction consistency regularization during early training to encourage consistency in scene structure. The experiments include baseline comparisons across different generation durations, component ablations, perturbation analysis, and human preference evaluation. The results show that AGF post-training using only five-second self-generated trajectories improves the quality of one-minute and four-minute videos while maintaining competitive short-video performance.
Critical strengths
-
Using distribution matching signals from future predictions to select history perturbations is a meaningful design choice. This connects the construction of challenging histories to the distribution matching objective and provides a clear rationale for selecting training perturbations. The paper explains the motivation and training procedure clearly. Improving long-horizon generation through post-training on short trajectories, while retaining the original four-step inference procedure, has practical value.
-
Different types of experiments provide complementary evidence of performance gains. AGF achieves the highest VBench Total score among the methods compared in the paper for one-minute generation, and its local quality is more stable over four-minute generation. Human evaluation also favors AGF, with a mean rank of 2.23 compared with 2.85 for the next-best method. These findings make me more confident that the improvements are perceptually meaningful, beyond the gains in automated metrics, and are the main reason I lean toward acceptance.
Critical weaknesses
While I find the full method effective, I have two main concerns.
-
A matched control is still needed to assess the additional benefit of training with history perturbations. The comparison with Gaussian perturbations of the same RMS magnitude supports the advantage of gradient-guided perturbations under the tested setting. However, the paper does not report an ablation that matches AGF’s main training settings but removes history perturbations. Comparisons with existing methods do not fully substitute for this control, leaving the additional benefit of this perturbation training scheme over matched training without perturbations unclear. Moreover, replacing the selected perturbations with Gaussian noise leads to a substantial performance drop. The paper reports an RMS perturbation budget of 0.05, but the rationale for this choice and whether similar degradation occurs at smaller budgets remain worth clarifying. These questions limit my understanding of where the method’s performance gains come from.
-
The actual evaluation configuration for four-minute whole-video consistency needs clarification. The paper distinguishes local quality from whole-video consistency and states that whole-video subject and background consistency scores include cross-clip comparisons. However, the released evaluation script leaves the cross-clip term disabled. I could not tell whether this was also the setting used for the tables. If so, the scores would measure consistency within individual clips, while changes in the subject or scene between clips could go unnoticed. The human rankings favor AGF, which I see as useful evidence of its practical value. My concern here is what the automatic consistency scores actually measure over the full four minutes.
Questions for authors
-
Are there ablation results using the same main training settings as AGF but without history perturbations? Also, how was the reported RMS perturbation budget of 0.05 chosen, and does replacing the selected perturbations with Gaussian noise still cause substantial degradation at smaller budgets?
-
Do the subject and background consistency scores in the four-minute experiments actually include cross-clip comparisons? Please clarify the evaluation settings used, explain the discrepancy between the paper’s description and the accompanying evaluation pipeline, and indicate whether it affects the reported values or the interpretation of the associated conclusions.
2 comments
Response to Reviewer V2cN (Part 2/2)
2. Evaluation code discrepancy and cross-clip score computation (W2 & Q2)
The reported consistency scores include cross-clip comparisons; the discrepancy arose because the supplementary package mistakenly contained an earlier evaluator that computes only within-clip consistency.
Thank you for identifying this packaging error. To clarify how the reported results were obtained, we describe the evaluation used for all six methods at 60s and 240s. In the official VBench-Long computation, each video's subject and background consistency scores combine a within-clip component with a mapped cross-clip component:
Here, denotes mean within-clip consistency. The raw cross-clip score is computed from the first frame of each two-second clip: each clip after the first is compared with both the first clip and its immediate predecessor, and the nonnegative cosine similarities are averaged.
The official mapping brings onto the within-clip score scale using predefined tables for subject and background consistency, with linear interpolation between entries. This mapping and the equal-weight fusion are applied to each video before scores are averaged across videos. Q5 uses the resulting fused subject and background consistency scores.
The following tables show each stage of this computation for the 240s results: within-clip scores, raw cross-clip scores, mapped scores, and the final fused scores.
\text{Method} & \begin{array}{c}\text{Self-}\cr\text{Forcing}\end{array} & \text{OPSD-V} & \text{SGF} & \text{LongLive} & \begin{array}{c}\text{Causal}\cr\text{Forcing}\end{array} & \text{AGF} \cr \hline \text{Within-clip }I & 0.9620 & 0.9586 & \mathbf{0.9712} & 0.9683 & 0.9370 & \quad 0.9690 \cr \text{Raw cross-clip }C & 0.7033 & 0.7184 & 0.7441 & 0.7712 & 0.6157 & \quad \mathbf{0.8015} \cr \text{Mapped }M(C) & 0.9651 & 0.9672 & 0.9710 & 0.9734 & 0.9519 & \quad \mathbf{0.9773} \cr \text{Fused }S & 0.9636 & 0.9629 & 0.9711 & 0.9708 & 0.9444 & \quad \mathbf{0.9732} \end{array}$$ > *R3-Table-5: Background consistency on 240-second videos (higher is better).* $$\begin{array}{l|rrrrrr} \text{Method} & \begin{array}{c}\text{Self-}\cr\text{Forcing}\end{array} & \text{OPSD-V} & \text{SGF} & \text{LongLive} & \begin{array}{c}\text{Causal}\cr\text{Forcing}\end{array} & \text{AGF} \cr \hline \text{Within-clip }I & 0.9532 & 0.9495 & 0.9579 & 0.9594 & 0.9379 & \quad \mathbf{0.9598} \cr \text{Raw cross-clip }C & 0.7771 & 0.7875 & 0.8007 & 0.8128 & 0.7133 & \quad \mathbf{0.8468} \cr \text{Mapped }M(C) & 0.9626 & 0.9633 & 0.9642 & 0.9654 & 0.9584 & \quad \mathbf{0.9678} \cr \text{Fused }S & 0.9579 & 0.9564 & 0.9610 & 0.9624 & 0.9482 & \quad \mathbf{0.9638} \end{array}$$ **AGF ranks highest in both fused consistency scores, which include the cross-clip contribution.** It also leads both raw cross-clip metrics, although the official mapping compresses the differences before fusion. We hope these experiments and clarifications resolve your questions about perturbation strength and the evaluation discrepancy. We will add the matched control and budget comparisons to the paper, clarify the score computation, and correct the supplementary material to match the evaluation used for the reported results.R3-Table-4: Subject consistency on 240-second videos (higher is better).
Response to Reviewer V2cN (Part 1/2)
Thank you for your constructive suggestions on the perturbation experiments and for pointing out the discrepancy in the supplementary evaluation code. We present the additional comparisons and clarify the cross-clip evaluation below.
1. Matched control and sensitivity to the perturbation budget (W1 & Q1)
History perturbations improve performance under the same training recipe. Reducing the Gaussian budget improves its scores, but AGF performs better at RMS 0.03 when both methods use no consistency regularization.
We first ran the requested control without history perturbations, keeping the remaining AGF training settings unchanged:
\text{Configuration} & \text{AGF} & \text{No perturbation} \cr \hline \text{5s Total} & \mathbf{82.03} & 81.41 \cr \text{60s Total} & \mathbf{81.47} & 76.34 \cr \text{240s Q5} & \mathbf{84.82} & 81.05 \end{array}$$ Removing history perturbations lowers the scores by **0.62, 5.13, and 3.77 points at 5s, 60s, and 240s**, respectively. Thus, the gains depend on history perturbations in addition to the shared training recipe. Regarding the choice of perturbation strength, we selected RMS 0.05 based on our experience in preliminary experiments. To address your question about smaller Gaussian budgets, we evaluated both perturbation types at RMS 0.03, 0.05, and 0.07. Consistency regularization, clean/perturbed loss weights, and other hyperparameters were held fixed across these runs. We extended all sweep runs from 400 to 600 training iterations to allow more time for convergence. > *R3-Table-2: AGF and Gaussian perturbations at three RMS budgets, with other training settings fixed (higher is better).* $$\begin{array}{l|c|c|c} \text{RMS budget} & 0.03 & 0.05 & 0.07 \cr \hline \begin{array}{l} \text{Perturbation} \cr \text{5s Total} \cr \text{60s Total} \cr \text{240s Q5} \end{array} & \begin{array}{rr} \text{AGF} & \text{Gauss} \cr 79.47 & 80.94 \cr 76.33 & 79.38 \cr 80.96 & 82.63 \end{array} & \begin{array}{rr} \text{AGF} & \text{Gauss} \cr \mathbf{81.68} & 74.32 \cr \mathbf{80.35} & 73.03 \cr \mathbf{85.36} & 79.23 \end{array} & \begin{array}{rr} \text{AGF} & \text{Gauss} \cr 78.25 & 77.33 \cr 76.85 & 75.47 \cr 81.28 & 78.52 \end{array} \end{array}$$ **Reducing the Gaussian budget from 0.05 to 0.03 improves all three scores**, so the degradation observed at 0.05 does not hold to the same extent at smaller budgets. At RMS 0.03, Gaussian perturbations also outperform AGF under the fixed regularization schedule. Since a regularization setting suited to one perturbation budget may be less suitable for another, we additionally evaluated both methods at RMS 0.03 with consistency regularization disabled: > *R3-Table-3: Effect of disabling prediction-consistency regularization for AGF and Gaussian perturbations at RMS 0.03 (higher is better).* $$\begin{array}{l|c|c} \text{Configuration} & \text{AGF (RMS = 0.03)} & \text{Gaussian (RMS = 0.03)} \cr \hline \begin{array}{l} \text{Regularization} \cr \text{5s Total} \cr \text{60s Total} \cr \text{240s Q5} \end{array} & \begin{array}{rr} \text{Default schedule} & \text{No regularization} \cr 79.47 & \mathbf{81.40} \cr 76.33 & 78.17 \cr 80.96 & \mathbf{84.34} \end{array} & \begin{array}{rr} \text{Default schedule} & \text{No regularization} \cr 80.94 & 81.19 \cr \mathbf{79.38} & 76.13 \cr 82.63 & 83.42 \end{array} \end{array}$$ With regularization disabled, **AGF outperforms Gaussian perturbations at all three durations at RMS 0.03**. These results show that reducing the budget makes Gaussian perturbations a stronger baseline, while the comparison also depends on the regularization setting. The RMS 0.05 AGF configuration in the sweep also scores higher than all tested Gaussian configurations.