acceptodds
Under review as a conference paper at ICLR 2027

Adversarial Gradient Forcing for Robust Few-Step Autoregressive Video Generation

Abstract

Autoregressive video generation models generate videos sequentially, with each new block conditioned on previously generated blocks. By training on the model's own rollouts, Self Forcing and its follow-up methods substantially mitigate exposure bias caused by the mismatch between training- and inference-time history distributions. However, these models still suffer from error accumulation during long-horizon generation. We identify a key underlying limitation: the model remains highly sensitive to errors in its generated history, allowing small deviations to be amplified through subsequent generations. We refer to this underexplored problem as the local history robustness gap. To address this gap, we introduce Adversarial Gradient Forcing (AGF), a framework designed to improve the robustness of few-step autoregressive video models to perturbations and accumulated errors in their own generated histories. AGF uses distribution matching gradients from future block predictions to select adversarial perturbations of self-generated histories. It then jointly trains the generator on the original and perturbed histories to improve the robustness of future generation. Extensive experiments show that AGF achieves the highest VBench Total score on 60-second video generation and the best visual quality on four-minute video generation. Notably, AGF achieves such long-horizon robustness while requiring only 5-second rollouts for training.

Primary area:
generative models
Keywords:
Video Generation, Autoregressive Diffusion Models, Distribution Matching Distillation, Adversarial Training, Long-Horizon Generation

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_V2cNOpenReview ↗
Rating
3
Confidence
3(+1)
Informativeness
2
Presentation
3
Soundness
3(+1)
Read review
Paper summary

This paper proposes Adversarial Gradient Forcing (AGF) to address error accumulation in few-step autoregressive video generation. The core idea is to use Distribution Matching Distillation (DMD) gradients from future predictions to construct targeted perturbations to model-generated history, and jointly optimize predictions conditioned on unperturbed and perturbed histories. The authors also introduce prediction consistency regularization during early training to encourage consistency in scene structure. The experiments include baseline comparisons across different generation durations, component ablations, perturbation analysis, and human preference evaluation. The results show that AGF post-training using only five-second self-generated trajectories improves the quality of one-minute and four-minute videos while maintaining competitive short-video performance.

Critical strengths
  1. Using distribution matching signals from future predictions to select history perturbations is a meaningful design choice. This connects the construction of challenging histories to the distribution matching objective and provides a clear rationale for selecting training perturbations. The paper explains the motivation and training procedure clearly. Improving long-horizon generation through post-training on short trajectories, while retaining the original four-step inference procedure, has practical value.

  2. Different types of experiments provide complementary evidence of performance gains. AGF achieves the highest VBench Total score among the methods compared in the paper for one-minute generation, and its local quality is more stable over four-minute generation. Human evaluation also favors AGF, with a mean rank of 2.23 compared with 2.85 for the next-best method. These findings make me more confident that the improvements are perceptually meaningful, beyond the gains in automated metrics, and are the main reason I lean toward acceptance.

Critical weaknesses

While I find the full method effective, I have two main concerns.

  1. A matched control is still needed to assess the additional benefit of training with history perturbations. The comparison with Gaussian perturbations of the same RMS magnitude supports the advantage of gradient-guided perturbations under the tested setting. However, the paper does not report an ablation that matches AGF’s main training settings but removes history perturbations. Comparisons with existing methods do not fully substitute for this control, leaving the additional benefit of this perturbation training scheme over matched training without perturbations unclear. Moreover, replacing the selected perturbations with Gaussian noise leads to a substantial performance drop. The paper reports an RMS perturbation budget of 0.05, but the rationale for this choice and whether similar degradation occurs at smaller budgets remain worth clarifying. These questions limit my understanding of where the method’s performance gains come from.

  2. The actual evaluation configuration for four-minute whole-video consistency needs clarification. The paper distinguishes local quality from whole-video consistency and states that whole-video subject and background consistency scores include cross-clip comparisons. However, the released evaluation script leaves the cross-clip term disabled. I could not tell whether this was also the setting used for the tables. If so, the scores would measure consistency within individual clips, while changes in the subject or scene between clips could go unnoticed. The human rankings favor AGF, which I see as useful evidence of its practical value. My concern here is what the automatic consistency scores actually measure over the full four minutes.

Questions for authors
  1. Are there ablation results using the same main training settings as AGF but without history perturbations? Also, how was the reported RMS perturbation budget of 0.05 chosen, and does replacing the selected perturbations with Gaussian noise still cause substantial degradation at smaller budgets?

  2. Do the subject and background consistency scores in the four-minute experiments actually include cross-clip comparisons? Please clarify the evaluation settings used, explain the discrepancy between the paper’s description and the accompanying evaluation pipeline, and indicate whether it affects the reported values or the interpretation of the associated conclusions.

2 comments
AuthorsOpenReview ↗

Response to Reviewer V2cN (Part 1/2)

Thank you for your constructive suggestions on the perturbation experiments and for pointing out the discrepancy in the supplementary evaluation code. We present the additional comparisons and clarify the cross-clip evaluation below.


1. Matched control and sensitivity to the perturbation budget (W1 & Q1)

History perturbations improve performance under the same training recipe. Reducing the Gaussian budget improves its scores, but AGF performs better at RMS 0.03 when both methods use no consistency regularization.

We first ran the requested control without history perturbations, keeping the remaining AGF training settings unchanged:

R3-Table-1: Effect of history perturbations under the AGF training recipe (higher is better).

\text{Configuration} & \text{AGF} & \text{No perturbation} \cr \hline \text{5s Total} & \mathbf{82.03} & 81.41 \cr \text{60s Total} & \mathbf{81.47} & 76.34 \cr \text{240s Q5} & \mathbf{84.82} & 81.05 \end{array}$$ Removing history perturbations lowers the scores by **0.62, 5.13, and 3.77 points at 5s, 60s, and 240s**, respectively. Thus, the gains depend on history perturbations in addition to the shared training recipe. Regarding the choice of perturbation strength, we selected RMS 0.05 based on our experience in preliminary experiments. To address your question about smaller Gaussian budgets, we evaluated both perturbation types at RMS 0.03, 0.05, and 0.07. Consistency regularization, clean/perturbed loss weights, and other hyperparameters were held fixed across these runs. We extended all sweep runs from 400 to 600 training iterations to allow more time for convergence. > *R3-Table-2: AGF and Gaussian perturbations at three RMS budgets, with other training settings fixed (higher is better).* $$\begin{array}{l|c|c|c} \text{RMS budget} & 0.03 & 0.05 & 0.07 \cr \hline \begin{array}{l} \text{Perturbation} \cr \text{5s Total} \cr \text{60s Total} \cr \text{240s Q5} \end{array} & \begin{array}{rr} \text{AGF} & \text{Gauss} \cr 79.47 & 80.94 \cr 76.33 & 79.38 \cr 80.96 & 82.63 \end{array} & \begin{array}{rr} \text{AGF} & \text{Gauss} \cr \mathbf{81.68} & 74.32 \cr \mathbf{80.35} & 73.03 \cr \mathbf{85.36} & 79.23 \end{array} & \begin{array}{rr} \text{AGF} & \text{Gauss} \cr 78.25 & 77.33 \cr 76.85 & 75.47 \cr 81.28 & 78.52 \end{array} \end{array}$$ **Reducing the Gaussian budget from 0.05 to 0.03 improves all three scores**, so the degradation observed at 0.05 does not hold to the same extent at smaller budgets. At RMS 0.03, Gaussian perturbations also outperform AGF under the fixed regularization schedule. Since a regularization setting suited to one perturbation budget may be less suitable for another, we additionally evaluated both methods at RMS 0.03 with consistency regularization disabled: > *R3-Table-3: Effect of disabling prediction-consistency regularization for AGF and Gaussian perturbations at RMS 0.03 (higher is better).* $$\begin{array}{l|c|c} \text{Configuration} & \text{AGF (RMS = 0.03)} & \text{Gaussian (RMS = 0.03)} \cr \hline \begin{array}{l} \text{Regularization} \cr \text{5s Total} \cr \text{60s Total} \cr \text{240s Q5} \end{array} & \begin{array}{rr} \text{Default schedule} & \text{No regularization} \cr 79.47 & \mathbf{81.40} \cr 76.33 & 78.17 \cr 80.96 & \mathbf{84.34} \end{array} & \begin{array}{rr} \text{Default schedule} & \text{No regularization} \cr 80.94 & 81.19 \cr \mathbf{79.38} & 76.13 \cr 82.63 & 83.42 \end{array} \end{array}$$ With regularization disabled, **AGF outperforms Gaussian perturbations at all three durations at RMS 0.03**. These results show that reducing the budget makes Gaussian perturbations a stronger baseline, while the comparison also depends on the regularization setting. The RMS 0.05 AGF configuration in the sweep also scores higher than all tested Gaussian configurations.
AuthorsOpenReview ↗

Response to Reviewer V2cN (Part 2/2)

2. Evaluation code discrepancy and cross-clip score computation (W2 & Q2)

The reported consistency scores include cross-clip comparisons; the discrepancy arose because the supplementary package mistakenly contained an earlier evaluator that computes only within-clip consistency.

Thank you for identifying this packaging error. To clarify how the reported results were obtained, we describe the evaluation used for all six methods at 60s and 240s. In the official VBench-Long computation, each video's subject and background consistency scores combine a within-clip component with a mapped cross-clip component:

S=0.5I+0.5M(C),S=0.5 I+0.5 M(C),

Here, II denotes mean within-clip consistency. The raw cross-clip score CC is computed from the first frame of each two-second clip: each clip after the first is compared with both the first clip and its immediate predecessor, and the nonnegative cosine similarities are averaged.

The official mapping MM brings CC onto the within-clip score scale using predefined tables for subject and background consistency, with linear interpolation between entries. This mapping and the equal-weight fusion are applied to each video before scores are averaged across videos. Q5 uses the resulting fused subject and background consistency scores.

The following tables show each stage of this computation for the 240s results: within-clip scores, raw cross-clip scores, mapped scores, and the final fused scores.

R3-Table-4: Subject consistency on 240-second videos (higher is better).

\text{Method} & \begin{array}{c}\text{Self-}\cr\text{Forcing}\end{array} & \text{OPSD-V} & \text{SGF} & \text{LongLive} & \begin{array}{c}\text{Causal}\cr\text{Forcing}\end{array} & \text{AGF} \cr \hline \text{Within-clip }I & 0.9620 & 0.9586 & \mathbf{0.9712} & 0.9683 & 0.9370 & \quad 0.9690 \cr \text{Raw cross-clip }C & 0.7033 & 0.7184 & 0.7441 & 0.7712 & 0.6157 & \quad \mathbf{0.8015} \cr \text{Mapped }M(C) & 0.9651 & 0.9672 & 0.9710 & 0.9734 & 0.9519 & \quad \mathbf{0.9773} \cr \text{Fused }S & 0.9636 & 0.9629 & 0.9711 & 0.9708 & 0.9444 & \quad \mathbf{0.9732} \end{array}$$ > *R3-Table-5: Background consistency on 240-second videos (higher is better).* $$\begin{array}{l|rrrrrr} \text{Method} & \begin{array}{c}\text{Self-}\cr\text{Forcing}\end{array} & \text{OPSD-V} & \text{SGF} & \text{LongLive} & \begin{array}{c}\text{Causal}\cr\text{Forcing}\end{array} & \text{AGF} \cr \hline \text{Within-clip }I & 0.9532 & 0.9495 & 0.9579 & 0.9594 & 0.9379 & \quad \mathbf{0.9598} \cr \text{Raw cross-clip }C & 0.7771 & 0.7875 & 0.8007 & 0.8128 & 0.7133 & \quad \mathbf{0.8468} \cr \text{Mapped }M(C) & 0.9626 & 0.9633 & 0.9642 & 0.9654 & 0.9584 & \quad \mathbf{0.9678} \cr \text{Fused }S & 0.9579 & 0.9564 & 0.9610 & 0.9624 & 0.9482 & \quad \mathbf{0.9638} \end{array}$$ **AGF ranks highest in both fused consistency scores, which include the cross-clip contribution.** It also leads both raw cross-clip metrics, although the official mapping compresses the differences before fusion. We hope these experiments and clarifications resolve your questions about perturbation strength and the evaluation discrepancy. We will add the matched control and budget comparisons to the paper, clarify the score computation, and correct the supplementary material to match the evaluation used for the reported results.
Reviewer_8KZjOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper identifies the local history robustness gap that contributes to error accumulation in few-step long-video generation, where the model is highly sensitive to errors in the generated history. To address this gap, the authors propose adversarial gradient forcing, which trains the model on adversarially perturbed videos to improve robustness and long-horizon generation quality.

Critical strengths

Strength 1: The paper addresses an important problem: error accumulation remains a critical challenge in long-video generation. The empirical results appear strong and demonstrate the effectiveness of the proposed approach.

Strength 2: The paper offers a novel and interesting perspective by linking error accumulation to the model’s sensitivity to perturbations in the generated history. This perspective naturally motivates a gradient-based analysis and the proposed adversarial training strategy.

Critical weaknesses

Weakness 1: The central argument appears somewhat circular. Error accumulation already describes how small errors in the generated history compound over time and degrade subsequent rollouts, which closely resembles sensitivity to small perturbations. The paper should clarify what additional explanatory insight the proposed robustness perspective provides.

Weakness 2: The paper lacks several important ablation studies. Please refer to the questions.

Questions for authors

Question 1: How does the method perform without the prediction consistency regularizer? Have you varied the weight of the adversarial loss in Eq. (2), and how sensitive is the method to this choice?

Question 2: Can the proposed adversarial perturbation strategy be applied to loss components other than the DMD loss? For example, ASD [1] introduces an auxiliary adversarial loss to distinguish between outputs generated in n steps and n+1 steps, while One-Forcing [2] uses an adversarial loss to distinguish clean videos from generated videos. The proposed strategy appears potentially applicable to these objectives as well. Could computing perturbations using an alternative objective, such as the adversarial loss in One-Forcing, yield further improvements?

[1] Yang, Yongqi, et al. “Towards One-Step Causal Video Generation via Adversarial Self-Distillation.” International Conference on Learning Representations, 2026.

[2] Feng, Jiaqi, et al. “One-Forcing: Towards Stable One-Step Autoregressive Video Generation.” arXiv preprint arXiv:2605.23458, 2026.

3 comments
AuthorsOpenReview ↗

Response to Reviewer 8KZj (Part 1/2)

We appreciate your positive assessment and your questions about the motivation and generality of AGF. Our response clarifies the role of history robustness and examines it through controlled experiments, ablations, and alternative training objectives.


1. Clarifying the role of history sensitivity (W1)

Viewing history as an input for adversarial training and studying the smoothness and robustness of the model's prediction is a natural but nontrivial perspective. We summarize the key difference as follow:

  • Existing methods study the problem that small errors consistently accumulate with timestep tt increasing;
  • AGF further studies that problem that even at a fixed timestep tt, small perturbations to the exactly same history could induce large variance in subsequent predictions, and amplify error accumulation.

By training on adversarially perturbed histories, AGF encourages the model’s predictions to vary more smoothly with the conditioning history, while steering the outputs toward a higher-quality distribution.

We demonstrate this robustness below through controlled measurements of generation quality and output variation.

We first compared how Self-Forcing and AGF respond to the same history perturbations. Both models generated continuations from identical unperturbed histories, then from the same histories with identical positive and negative perturbations applied. For each model, we measured continuation Q5, the mean quality drop relative to its own unperturbed continuation, and the variance of these quality changes across perturbations:

R2-Table-1: Continuation quality under identical history perturbations. Variance is measured across positive and negative perturbations, in squared Q5 points.

\text{Method} & \begin{array}{c}\text{Unperturbed}\cr\text{Q5}\uparrow\end{array} & \begin{array}{c}\text{Perturbed}\cr\text{Q5}\uparrow\end{array} & \begin{array}{c}\text{Mean Q5}\cr\text{drop}\downarrow\end{array} & \begin{array}{c}\text{Variance of}\cr\text{Q5 changes}\downarrow\end{array} \cr \hline \text{Self-Forcing} & 83.604 & 81.985 & 1.618 & 3.241 \cr \text{AGF} & \mathbf{85.138} & \mathbf{84.361} & \mathbf{0.776} & \mathbf{0.962} \cr \hline \text{AGF improvement} & +1.534 & +2.376 & -52.0\\% & -70.3\\% \end{array}$$ **AGF maintains higher quality, with 52.0% less quality loss and 70.3% lower variance under the same perturbations.** These results support our analysis that history robustness helps preserve generation quality under input perturbations. We also measured the latent output $x_0$ before VAE decoding under eight perturbations (four orthogonal directions, each with both signs), averaging over six five-second windows of a 30-second continuation. We report perturbation amplification (output deviation divided by input perturbation magnitude) and variance across the eight outputs: > *R2-Table-2: Response of generated latents before VAE decoding (lower is better). Relative reductions are reported with 95% confidence intervals.* $$\begin{array}{l|rrrc} \text{Metric} & \text{Self-Forcing} & \text{AGF} & \text{Reduction} & \text{95\\% CI} \cr \hline \text{Perturbation amplification} & 2.9459 & \mathbf{2.5618} & 13.0\\% & [9.9\\%, 16.3\\%] \cr \text{Conditional variance} & 0.3495 & \mathbf{0.2611} & 25.3\\% & [19.3\\%, 31.2\\%] \end{array}$$ Both measures are lower for AGF, demonstrating **a smoother response to history changes alongside the higher generation quality shown above**.
AuthorsOpenReview ↗

Response to Reviewer 8KZj (Part 2/2)

2. Additional ablation results (W1 & Q1)

The ablations support our regularization design and the 50:50 loss ratio.

The paper includes an ablation without prediction consistency regularization (“None” in the ablation table), reproduced below:

R2-Table-3: Effect of prediction consistency regularization (higher is better).

\text{Configuration} & \text{AGF} & \begin{array}{c}\text{Without consistency}\cr\text{regularization}\end{array} \cr \hline \text{5s Total} & \mathbf{82.03} & 81.38 \cr \text{60s Total} & \mathbf{81.47} & 77.96 \cr \text{240s Q5} & \mathbf{84.82} & 82.75 \end{array}$$ Even without the regularizer, AGF achieves a strong **240s Q5 of 82.75**. Under the current training budget, adding the regularizer further improves performance at all three durations, including gains of **3.51 points at 60s and 2.07 points at 240s**. The clean-to-adversarial ratio controls the relative weights of the two DMD loss terms in our training objective. We selected 50:50 through early experiments. To further examine the effect of this choice, we provide an additional ablation comparing three ratios at RMS 0.05 and 400 training iterations, with all other settings fixed: > *R2-Table-4: Effect of the clean-to-adversarial loss ratio with other training settings fixed (higher is better).* $$\begin{array}{l|rrr} \text{Clean:adversarial} & 60{:}40 & 50{:}50 & 40{:}60 \cr \hline \text{5s Total} & 81.80 & \mathbf{82.03} & 73.44 \cr \text{60s Total} & 79.31 & \mathbf{81.47} & 72.76 \cr \text{240s Q5} & 82.79 & \mathbf{84.82} & 78.73 \end{array}$$ **The 50:50 ratio gives the best results in this comparison.** Considering that the fake-score model's clean/adversarial sampling ratio and the regularization parameters were kept fixed, tuning these settings for each generator loss ratio may yield better performance. We have not conducted this joint tuning due to limited time and resources. --- ### ***3. Alternative objectives for history perturbation selection (Q2)*** **Replacing DMD as the perturbation-selection objective did not yield further gains over the original AGF. At 240s, the ASD-based AGF variant remained comparable to Self-Forcing, while the One-Forcing-based variant improved Q5 by 0.76 points.** Following your suggestion, we explored AGF variants that use adversarial objectives from ASD and One-Forcing to select history perturbations, while retaining DMD in generator training: > *R2-Table-5: Exploratory AGF variants using alternative adversarial objectives (higher is better).* $$\begin{array}{l|rrr} \text{Configuration} & \begin{array}{c}\text{Self-}\cr\text{Forcing}\end{array} & \begin{array}{c}\text{AGF}\cr\text{(ASD)}\end{array} & \begin{array}{c}\text{AGF}\cr\text{(One-Forcing)}\end{array} \cr \hline \text{5s Total} & \mathbf{82.21} & 81.32 & 81.55 \cr \text{60s Total} & \mathbf{79.56} & 78.23 & 76.67 \cr \text{240s Q5} & 82.47 & 82.44 & \mathbf{83.23} \end{array}$$ Notably, the auxiliary adversarial gradients may conflict with the DMD training gradients early in training. This motivates a more conservative schedule to stabilize training and make better use of the additional signal. --- We hope these responses address your concerns. We will incorporate the additional experiments discussed above, together with further explanations of AGF's motivation, mechanism, and underlying principles, into the main text.

Thanks for your rebuttal! my concerns are addressed. I am happy to keep my score.

Reviewer_FRpvOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies error accumulation in few-step autoregressive video generation and identifies sensitivity to imperfect self-generated histories as a remaining limitation of on-policy training. The proposed Adversarial Gradient Forcing constructs bounded history perturbations by backpropagating distribution-matching gradients from subsequent block predictions. The generator is then optimized under both original and perturbed histories, with an early-stage prediction consistency regularizer to preserve scene structure. On a Wan2.1-1.3B generator, AGF improves 60-second VBench performance and four-minute visual quality while training exclusively on five-second rollouts.

Critical strengths
  1. This work makes a useful distinction between exposure to self-generated histories and robustness to errors within those histories. The controlled history interventions, including the recovery experiments in Appendix G.2, support the practical relevance of this distinction.

  2. Using future distribution-matching gradients gives the perturbation selection a quality-related objective. Appendix A.2 clearly explains why maximizing prediction differences alone need not identify harmful changes. The paired training objective and first-step consistency regularizer form a coherent approach to improving robustness while preserving scene structure.

  3. AGF achieves the highest 60-second Total score among the evaluated methods (81.47) and the highest four-minute Q5 (84.82), with approximately preserved five-second performance. The temporal quality curves and human study provide complementary evidence: AGF achieves a mean human rank of 2.23, compared with 2.85 for the next-best method.

  4. Obtaining long-horizon improvements from five-second rollouts is appealing, and the reported post-initialization training requires no additional real-video data. The appendices document gradient paths, evaluation protocols, training schedules, and cost-accounting assumptions in useful detail. The alternative Causal Forcing initialization also provides some evidence that the gains are not specific to one starting checkpoint.

Critical weaknesses
  1. The ablations do not fully isolate adversarial selection from the training recipe. Table 2 demonstrates that both perturbation choice and regularization matter, but a clean-only DMD control with the same initialization, optimizer, replay, EMA, and generator-update schedule would make the attribution clearer. Moreover, the Gaussian baseline performs substantially worse (75.11 vs 84.82 Q5 at four minutes), but it is tested at the same latent RMS budget rather than with a separately tuned strength. Equal RMS does not necessarily imply comparable semantic corruption after decoding. The reported comparison therefore establishes the advantage at the chosen budget, while leaving the strength of a tuned non-adversarial alternative uncertain.

  2. The link between the selection surrogate and long-horizon robustness remains partly indirect. Appendix B.2 clarifies that perturbations are selected through parallel predictions with recorded histories, without inserting newly predicted blocks into subsequent histories. This is a reasonable efficiency choice, but it differs from recursive error propagation during inference. Appendix A explains the propagation mechanism and establishes optimality for a local first-order criterion; it does not establish that optimizing this surrogate reduces long-horizon propagation gains. The intervention experiments are helpful, but a quantitative connection between the selection score and actual downstream degradation would strengthen the central explanation.

  3. The temporal metric supports a narrower claim than comprehensive long-term consistency. As Appendix D.3 acknowledges, local Q5 measures quality within successive intervals and does not establish correspondence to the opening scene. The whole-video scores, qualitative examples, and human rankings help address this limitation, but maintaining local visual quality can still coexist with gradual changes in identity, composition, or prompt adherence. More targeted long-range consistency measurements would clarify what aspects of four-minute generation are stabilized.

Questions for authors
  1. Could the authors provide a clean-only DMD control using the same training recipe and generator-update count, together with a perturbation-strength sweep for AGF and the Gaussian baseline? Does the advantage of gradient-based selection persist when each method uses its own best perturbation budget?

  2. How well does the first-order selection score predict the degradation caused by an actual recursive continuation? For example, across sampled history blocks and perturbation directions, is a larger predicted DMD increase associated with a larger subsequent quality drop? A small comparison with selection through a short recursive continuation would also help quantify the approximation in Appendix B.2.

  3. Could the authors supplement local Q5 with explicit early-to-late identity or scene consistency and prompt-adherence measurements, preferably separating mostly static scenes from scenes requiring sustained action? This would help establish how broadly the observed stability extends beyond local visual quality.

  4. Could the authors report paired confidence intervals for the main AGF-vs-SGF and AGF-vs-LongLive differences, and assess sensitivity to additional generation seeds? Even a limited repeated-training experiment would help determine whether the main gains and the preferred regularization configuration are reproducible.

2 comments
AuthorsOpenReview ↗

Response to Reviewer FRpv (Part 1/2)

Thank you for your careful review. Below, we address the experimental comparisons, selection criterion, and long-video performance with additional evidence.


1. Contribution of history perturbations and comparison across budgets (W1)

History perturbations improve AGF's performance, and AGF outperforms Gaussian perturbations when each uses its best tested budget.

We isolated the contribution of history perturbations by removing them while keeping the rest of the training recipe unchanged:

R1-Table-1: Effect of history perturbations under the AGF training recipe (higher is better).

\text{Configuration} & \text{AGF} & \text{No perturbation} \cr \hline \text{5s Total} & \mathbf{82.03} & 81.41 \cr \text{60s Total} & \mathbf{81.47} & 76.34 \cr \text{240s Q5} & \mathbf{84.82} & 81.05 \end{array}$$ The gains are larger at 60s and 240s. We next compared adversarial and Gaussian perturbations, allowing each to use its best tested budget. We swept three RMS budgets with consistency regularization, clean/perturbed loss weights, and other hyperparameters fixed. All sweep runs used 600 rather than 400 training iterations to allow more time for convergence. > *R1-Table-2: AGF and Gaussian perturbations at three RMS budgets, with other training settings fixed (higher is better).* $$\begin{array}{l|c|c|c} \text{RMS budget} & 0.03 & 0.05 & 0.07 \cr \hline \begin{array}{l} \text{Perturbation} \cr \text{5s Total} \cr \text{60s Total} \cr \text{240s Q5} \end{array} & \begin{array}{rr} \text{AGF} & \text{Gauss} \cr 79.47 & 80.94 \cr 76.33 & 79.38 \cr 80.96 & 82.63 \end{array} & \begin{array}{rr} \text{AGF} & \text{Gauss} \cr \mathbf{81.68} & 74.32 \cr \mathbf{80.35} & 73.03 \cr \mathbf{85.36} & 79.23 \end{array} & \begin{array}{rr} \text{AGF} & \text{Gauss} \cr 78.25 & 77.33 \cr 76.85 & 75.47 \cr 81.28 & 78.52 \end{array} \end{array}$$ AGF performs best at RMS 0.05 and Gaussian perturbations at 0.03; AGF leads at all three durations at these respective budgets. Since the fixed regularization and other settings may need adjustment as the budget changes, we also compared both methods at RMS 0.03 without consistency regularization: > *R1-Table-3: Effect of disabling prediction-consistency regularization for AGF and Gaussian perturbations at RMS 0.03 (higher is better).* $$\begin{array}{l|c|c} \text{Configuration} & \text{AGF (RMS = 0.03)} & \text{Gaussian (RMS = 0.03)} \cr \hline \begin{array}{l} \text{Regularization} \cr \text{5s Total} \cr \text{60s Total} \cr \text{240s Q5} \end{array} & \begin{array}{rr} \text{Default schedule} & \text{No regularization} \cr 79.47 & \mathbf{81.40} \cr 76.33 & 78.17 \cr 80.96 & \mathbf{84.34} \end{array} & \begin{array}{rr} \text{Default schedule} & \text{No regularization} \cr 80.94 & 81.19 \cr \mathbf{79.38} & 76.13 \cr 82.63 & 83.42 \end{array} \end{array}$$ **AGF at RMS 0.05 leads all tested Gaussian configurations at all three durations.** At RMS 0.03, AGF also leads when both methods omit regularization, supporting adversarial selection under matched settings. --- ### ***2. Reliability of the first-order selection score (W2 & Q1)*** **Higher first-order scores tend to predict greater quality loss; parallel selection finds harmful perturbations at lower cost than recursive selection.** Using our reproduced Self-Forcing checkpoint, we evaluated five perturbation directions at the same budget. We compared predicted DMD increases with quality drops over the first five seconds of recursive continuation. The mean Spearman correlation was **0.282 (95% CI: [0.095, 0.465])**. We then compared parallel and recursive selection at the same perturbation budget, measuring the quality drops in actual recursive continuation: > *R1-Table-4: Quality drops caused by different perturbation directions.* $$\begin{array}{l|rrcc} \text{Direction} & \text{Q5} & \begin{array}{c}\text{Q5}\cr\text{drop}\end{array} & \begin{array}{c}\text{95\\% CI}\cr\text{of drop}\end{array} & \begin{array}{c}\text{Relative}\cr\text{time}\end{array} \cr \hline \text{Parallel (ours)} & 83.946 & 0.308 & [0.160, 0.454] & \mathbf{58\\%} \cr \text{Recursive} & 83.866 & 0.387 & [0.065, 0.715] & 100\\% \end{array}$$ **The parallel direction produces about 80% of the recursive direction's mean Q5 drop while taking 58% of the time to compute.** We also measured the RMS change in the generated latents relative to the unperturbed continuation: > *R1-Table-5: Latent deviation of recursive continuations from the unperturbed output.* $$\begin{array}{l|rc} \text{Direction} & \text{RMS deviation} & \text{95\\% CI} \cr \hline \text{Parallel (ours)} & 0.342 & [0.293, 0.391] \cr \text{Recursive} & 0.531 & [0.453, 0.617] \end{array}$$ The parallel direction causes substantial latent changes and quality loss at lower computational cost.
AuthorsOpenReview ↗

Response to Reviewer FRpv (Part 2/2)

3. Clarification of cross-clip evaluation (W3)

The reported scores include cross-clip comparisons, and AGF's advantage is more pronounced in the raw cross-clip results.

We will clarify the original wording. Below are the cross-clip computation and original score components.

We use the official VBench-Long computation and score mapping for all six methods at 60s and 240s:

S=0.5I+0.5M(C),S=0.5 I+0.5 M(C),

where II is mean within-clip consistency. For CC, we take the first frame of each two-second clip and compare each clip after the first with both the first and the preceding clip. Averaging these nonnegative cosine similarities gives the video's raw cross-clip score.

To make the components comparable, MM maps CC onto the within-clip scale using the official subject and background mapping tables, with linear interpolation between entries. We map and fuse each video’s scores before averaging across videos.

The tables below report the 240s score components. Q5 incorporates the fused subject and background consistency scores.

R1-Table-6: Subject consistency on 240-second videos (higher is better).

\text{Method} & \begin{array}{c}\text{Self-}\cr\text{Forcing}\end{array} & \text{OPSD-V} & \text{SGF} & \text{LongLive} & \begin{array}{c}\text{Causal}\cr\text{Forcing}\end{array} & \text{AGF} \cr \hline \text{Within-clip }I & 0.9620 & 0.9586 & \mathbf{0.9712} & 0.9683 & 0.9370 & \quad 0.9690 \cr \text{Raw cross-clip }C & 0.7033 & 0.7184 & 0.7441 & 0.7712 & 0.6157 & \quad \mathbf{0.8015} \cr \text{Mapped }M(C) & 0.9651 & 0.9672 & 0.9710 & 0.9734 & 0.9519 & \quad \mathbf{0.9773} \cr \text{Fused }S & 0.9636 & 0.9629 & 0.9711 & 0.9708 & 0.9444 & \quad \mathbf{0.9732} \end{array}$$ > *R1-Table-7: Background consistency on 240-second videos (higher is better).* $$\begin{array}{l|rrrrrr} \text{Method} & \begin{array}{c}\text{Self-}\cr\text{Forcing}\end{array} & \text{OPSD-V} & \text{SGF} & \text{LongLive} & \begin{array}{c}\text{Causal}\cr\text{Forcing}\end{array} & \text{AGF} \cr \hline \text{Within-clip }I & 0.9532 & 0.9495 & 0.9579 & 0.9594 & 0.9379 & \quad \mathbf{0.9598} \cr \text{Raw cross-clip }C & 0.7771 & 0.7875 & 0.8007 & 0.8128 & 0.7133 & \quad \mathbf{0.8468} \cr \text{Mapped }M(C) & 0.9626 & 0.9633 & 0.9642 & 0.9654 & 0.9584 & \quad \mathbf{0.9678} \cr \text{Fused }S & 0.9579 & 0.9564 & 0.9610 & 0.9624 & 0.9482 & \quad \mathbf{0.9638} \end{array}$$ **AGF leads both raw cross-clip consistency metrics and both fused scores.** However, the official mapping compresses these differences, making AGF's cross-clip advantage less apparent in the fused scores reported in the paper. For example, AGF and LongLive's subject scores are 0.8015 and 0.7712 before mapping, but 0.9773 and 0.9734 afterward. Due to limited time and resources, we have not separately evaluated static scenes and sustained actions. --- ### ***4. Paired confidence intervals and an additional generation seed*** **Paired confidence intervals support AGF's gains at 60s and 240s, and AGF maintains strong performance with an additional generation seed.** We quantify uncertainty in AGF's gains over SGF and LongLive using paired 95% confidence intervals: > *R1-Table-8: AGF minus baseline, in score points, with paired 95% confidence intervals.* $$\begin{array}{l|c|c} \text{Baseline} & \text{SGF} & \text{LongLive} \cr \hline \begin{array}{l} \text{Metric} \cr \text{5s Total} \cr \text{60s Total} \cr \text{240s Q5} \end{array} & \begin{array}{rr} \text{Difference} & \text{95\\% CI} \cr -0.05 & [-0.77, 0.65] \cr +1.80 & [0.73, 2.91] \cr +0.75 & [0.40, 1.10] \end{array} & \begin{array}{rr} \text{Difference} & \text{95\\% CI} \cr +1.69 & [0.86, 2.51] \cr +1.66 & [0.42, 2.85] \cr +1.21 & [0.83, 1.59] \end{array} \end{array}$$ At 5s, AGF is comparable to SGF and scores higher than LongLive. At 60s and 240s, **all four confidence intervals lie above zero**, supporting the long-video improvements over both baselines. We repeated evaluation with generation seed 42, keeping the trained checkpoint and other evaluation settings unchanged: > *R1-Table-9: AGF evaluated with an additional generation seed (higher is better).* $$\begin{array}{l|rr} \text{Evaluation} & \text{Original} & \text{Generation seed 42} \cr \hline \text{5s Total} & 82.03 & 82.73 \cr \text{60s Total} & 81.47 & 81.11 \cr \text{240s Q5} & 84.82 & 84.71 \end{array}$$ AGF maintains strong performance across all three durations with a different generation seed, with its 5s Total score increasing from the reported 82.03 to 82.73. --- We hope these responses address your concerns. We will add the controls, selection-score validation, and statistical and cross-clip results to the paper.

Comments on the paper

2 comments

We sincerely thank the reviewers for their valuable feedback. We address each concern point by point in the corresponding thread. We summarize two recurring concerns below:

  • Contribution of history perturbations and sensitivity to hyper-parameters. Our replies include a matched control without history perturbations, comparisons with Gaussian perturbations across RMS budgets, and ablations of prediction consistency regularization;
  • Evaluation of long-horizon consistency. We clarify what the four-minute metrics measure, how cross-clip comparisons contribute to the reported subject and background consistency scores, and why the supplementary evaluation script differed from the one used for the reported results.

Thank you for your consideration.

Area_Chair_wDMbOpenReview ↗

reviewer discussion

Dear Reviewers,

The authors have submitted their rebuttal. Please read their response carefully and engage in the discussion as soon as possible. Let us know whether the rebuttal addresses your concerns and update your final rating accordingly.

Thank you for your time and service to ICLR.

Best regards,

AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.