acceptodds
Withdrawn from ICLR 2027

Clean Forcing: Drift-Resistant Autoregressive Video Diffusion with a Frozen Base

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Autoregressive (AR) video diffusion models generate streaming video of arbitrary length by conditioning each new chunk of frames on those already generated. However, they suffer from exposure bias, where a model trained on clean context must continue from its own imperfect outputs, and errors compound until quality collapses within seconds (drift). Existing methods either require cluster-scale base retraining, apply training-free inference corrections with inconsistent results, or train frozen-base correctors for sampling and cache errors. We instead find that drift is largely deterministic and therefore learnable, as holding the noisy state fixed while varying only the history reveals a counterfactual velocity gap that is 95% systematic across noise seeds. Based on this observation, we introduce *Clean Forcing*, which trains a 5.9M-parameter LoRA corrector on the *frozen* base model to regress this gap, with a closed-loop stage that adds DAgger-style aggregation and a drift-contraction objective to expose the corrector to its own rollouts. Because the clean histories can be the base model's own single-shot generations, training needs *no real videos*, and the merged corrector adds no inference cost. In 50s text-to-video generation, Clean Forcing reduces -drift from to without real video and to with 40 real clips, improving on the best published result of with over less data. Extensive experiments demonstrate that Clean Forcing outperforms prior methods in aesthetic quality and data efficiency with reduced drift for long-horizon video generation.

Primary area:
generative models
Keywords:
autoregressive video diffusion, long video generation, drift correction, exposure bias, error accumulation, LoRA, parameter-efficient fine-tuning, flow matching, world models

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_uLvwOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper introduces Clean Forcing, a learned correction for drift in autoregressive video diffusion with a frozen base. It compares the base model's velocity predictions under clean and generated histories at the same noisy state, then trains a small LoRA corrector to match the clean-history prediction. DAgger-style data aggregation and a next-chunk drift-contraction loss expose the corrector to its own generated states. Experiments report substantial improvements in quality and drift on a causally adapted Wan2.1-1.3B base, with additional scaling and transfer studies.

Critical strengths
  1. A clear motivation to directly reduce accumulated errors. The paper aims to explicitly correct prediction errors caused by self-generated history, using predictions conditioned on reference history as supervision. This is an intuitive and appealing direction for addressing error accumulation in autoregressive video generation.
Critical weaknesses
  1. The training objective does not isolate error accumulation. The objective minimizes the difference between predicted velocities conditioned on self-generated history and pre-generated reference history. However, these histories can differ in semantic content and motion, even when they share the same starting point and caption. Their velocity difference therefore reflects both accumulated errors and legitimate differences between possible continuations. Matching the reference-history prediction may consequently penalize valid semantic or motion variation. This also raises a concern that the objective could favor near-static continuations that limit divergence from the reference, rather than correcting errors while preserving scene progression.

  2. Visible artifacts and missing long-video baselines Horizontal line artifacts are visible in some frames generated by the proposed method in Figure 4, suggesting that substantial visual artifacts remain despite the improvement in drift scores. The paper also does not include experimental comparisons with relevant long-video generation baselines such as LongLive and Rolling Forcing.

Questions for authors
  1. How do the authors distinguish velocity differences caused by accumulated errors from those caused by legitimate semantic or motion differences between the generated and reference histories? Could controlled experiments test whether the objective penalizes valid scene progression or encourages near-static continuations?

  2. What causes the horizontal line artifacts visible in Figure 4, and can they be reduced without sacrificing drift resistance? Could the authors compare against LongLive and Rolling Forcing under matched evaluation settings, reporting visual quality, temporal coherence, and drift?

Reviewer_fFjzOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper addresses drift in autoregressive (AR) video diffusion. It first measures the velocity gap between the model's predictions at the same noisy state under a clean versus a self-generated history, and finds it ~95% consistent across noise seeds. It then trains Clean Forcing, a rank-16 LoRA corrector (5.9M parameters) on the frozen base, to map the generated-history velocity to the clean-history velocity, followed by a closed-loop stage with DAgger-style aggregation and a drift-contraction loss. Clean histories can come from the base model's own bidirectional 5 s generations, so no real video is required, and the merged LoRA adds no inference cost. On a lightly causally adapted Wan2.1-1.3B, Clean Forcing reduces ∆-drift over 50 s from +11.27 to +1.65 (no real video) or −0.16 (40 real clips), with additional results on Wan2.1-14B, a Causal Forcing checkpoint, and two AR world models.

Critical strengths
  1. A clear and cheap idea. The counterfactual diagnostic (same noisy state, clean vs. generated history) motivates a well-defined regression target, and the resulting corrector is lightweight: 5.9M parameters, about one GPU-day, no real videos required, and no inference overhead after merging (+3.5% latency, Welch p = 0.40, Table 5). Measured per frame, it substantially reduces quality drift across several bases: Wan2.1-1.3B (+11.27 → +1.65 / −0.16), Wan2.1-14B (+15.65 → +0.97), and the Causal Forcing checkpoint (+11.99 → +2.80), and it also reduces drift on two other AR world models.
  2. Unusually transparent evaluation. The paper reports checks for degenerate outputs (Appendix C), the drift–progression tradeoff (Appendix D), a failure case on the distilled Self Forcing model (Appendix E), SEMs throughout, a horizon sweep, and a user study with catch trials. It also reports the temporal-consistency metrics (Table 11) that expose the weakness below.
Critical weaknesses
  1. The generated 50 s videos are not temporally continuous. In all qualitative results in the supplementary material, the Clean Forcing videos visibly jump at every chunk boundary (~0.75 s, i.e., every 3-latent chunk). Some content from the previous chunks carries over, mostly semantically, but to a viewer who does not know the method, the videos look like similar 0.75 s clips concatenated together rather than one long video with smooth transitions between every frame, which is a basic requirement of long video generation. The artifact is severe enough that it cannot be ignored, and it is introduced by the method: the adapted base (before it drifts), all baseline methods in the comparison videos, the AR base in the teaser, and the 14B base all generate smooth videos. The paper's own metrics agree: the +real variant has the worst temporal LPIPS (0.59) and flicker (32.8) in Table 11, versus 0.15 and 9.5 for Self Forcing; it has the lowest 2-second lagged identity (0.746 vs. 0.922 for Self Forcing, Table 15); and in the user study, raters show no preference over Self Forcing on temporal stability (46.9%) and prefer SkyReels-V2 (21.0% for Clean Forcing). The paper describes this as a "luminance pulse" and attributes it to chunked generation (Appendix G.2, L), but that is inconsistent with the smooth baselines that use the same chunk structure, and the proposed overlap decoding does not fix it in any of the shown results. Because ∆-drift (first-20% minus full-video MUSIQ) and VBench score frames largely independently, they cannot detect this failure. The method thus appears to avoid drift partly by re-rendering each chunk rather than continuing smoothly from the history, so the central claim of drift-resistant long video generation is not established.
  2. Limited applicability and overstated comparisons. (a) The method is demonstrated on lightly adapted bases: the main base has only 4K causal-adaptation steps on 300 synthetic clips (subject consistency 65.7 vs. 80.2 for Self Forcing; MUSIQ 44.8). On a base adapted for 20K steps, the corrected result is worse (MUSIQ 62.3 vs. 66.9), and on the distilled Self Forcing model, the corrector increases drift from +1.08 to +8.3 (Appendix E). No strong released checkpoint is shown to benefit, which undercuts the motivation that "deployed checkpoints still lack an efficient learned correction". (b) The abstract's "improving on the best published result of +3.57 with over 150× less data" compares against BAgger on a different base and prompt set, which the paper itself calls "indicative"; on the same prompts, Self Forcing (+1.08) and SkyReels-V2 (+1.00) have lower drift than the zero-real variant (+1.65). (c) ∆-drift rewards outputs that stay close to their opening: the +real variant's anchoring is 0.795 vs. 0.485 for Self Forcing (Table 15). (d) The ~95% systematic fraction α* varies only the noise seed while holding both histories fixed; since the corrector never sees the clean history, this does not by itself show that the gap is learnable from inference-time inputs (the validation R², 0.85 at 1.3B and 0.48 at 14B, is the relevant evidence). (e) The clean history is the reference clip, which shares only its first 3 latents with the rollout (Appendix A), so the target also pushes toward another video's content; this may explain both the anchoring and the per-chunk re-rendering.
Questions for authors
  1. (Experiment) Can you report a continuity metric specific to chunk boundaries (e.g., frame difference or flow-warping error at boundaries vs. within chunks) for all methods, including the 14B and Causal Forcing results?
  2. (Experiment) Does the corrector reduce the model's dependence on the generated history, e.g., attention mass on history tokens, or sensitivity of the output to perturbing the history?
  3. (Experiment) Can the discontinuity be removed while keeping the drift reduction? For example, does overlap decoding (Appendix G.2) remove the structural jump or only the brightness component, and does using clean references that are continuations of the rollout itself (rather than a different clip sharing the first 3 latents) avoid it?
  4. (Experiment) Is there a strong released checkpoint, beyond the lightly adapted bases and the teacher-forced Causal Forcing checkpoint, on which Clean Forcing improves drift without harming continuity?
  5. (Clarification) Please revise the abstract's comparison to BAgger and state the same-prompt comparison with Self Forcing and SkyReels-V2 explicitly.
Reviewer_3aKZOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Clean Forcing addresses quality drift in autoregressive video diffusion by training a lightweight corrector on a frozen, causally adapted base. At a shared noisy state, the method matches a LoRA student's prediction under generated history to the frozen model's prediction under clean reference history. A second stage aggregates rollouts generated with the corrector active and adds a next-chunk loss whose gradient flows through the current clean-latent estimate. On 128 held-out prompts at 50 seconds, the method improves its adapted Wan2.1-1.3B base from 44.8 to 66.9 MUSIQ and reduces Δ-drift from +11.27 to +1.65 using synthetic references. Additional experiments assess a 14B base, an independently trained Causal-Forcing checkpoint, other world models, inference cost, and temporal limitations.

Critical strengths
  1. The paper proposes a simple corrective objective that produces substantial improvements without changing the model architecture. The work demonstrates a practical way to improve an existing generator without full retraining or an additional inference network. The synthetic-reference result is particularly useful for settings with limited task-specific video data.
  2. The evaluation extends beyond a single favorable metric and explicitly reports important failure cases, which include quality-over-time curves, alternative drift diagnostics, short-horizon checks, ablations, human judgments, and extensive supplementary videos.
Critical weaknesses
  1. The headline metric does not establish that drift is corrected, and temporal coherence gets worse. Δ\Delta-drift is a per-frame image-quality difference (MUSIQ), so it can be satisfied by freezing the composition and re-sampling content. The paper’s own diagnostics point that way for the +40-real variant used in the headline and in the user study: ◦ It has the highest anchoring (0.795 vs. 0.485 for Self Forcing). ◦ It has the lowest 2-s lagged identity of all methods, including the uncorrected base (0.746 vs. 0.812 base, 0.922 Self Forcing, 0.927 SkyReels; Table 15). ◦ It has the worst t-LPIPS and flicker (0.59 / 32.8 vs. 0.15 / 9.5 for Self Forcing; Table 11), plus a luminance pulse at chunk cadence.. ◦ Raters do not prefer it to Self Forcing on temporal stability (46.9%) and clearly prefer SkyReels (21.0%).. The supplementary videos match this. In p008 (rally car) the framing stays almost fixed for 50 s while the car’s body shape and livery keep changing across the rollout. In p101 the clock face (numerals, layout) changes, with ghosted transitions. So “−0.16, no measurable quality decay” and “drift reduction does not trade off overall quality” (§4.1) overstate the result. Coherence metrics belong in Table 1, not Appendices D/I. The zero-real variant behaves better (lagged identity 0.830; anchoring 0.679 ≈ SkyReels), so the claims might rest more safely on it.
  2. The comparisons and scope do not support “matches cluster-scale retraining with 150× less data” as stated.

◦ (a) The 150× figure and “improves on the best published +3.57” come from BAgger numbers on a different base and prompt set. The paper calls this comparison indicative, yet it appears in the abstract and TL;DR.. ◦ (b) The most relevant released long-horizon baselines on the same Wan2.1-1.3B backbone, LongLive and Rolling Forcing, are cited but not evaluated. Self Forcing was trained for short clips and is a weak 50-s reference.. ◦ (c) All positive results are on multi-step bases that drift heavily under the authors’ protocol. One is their own 4K-step adaptation. The other is the Causal-Forcing teacher run with a non-default 20-step, no-sink setup, where a half-window sink alone already moves it from +11.07 to +5.29 (Table 8). On the strong distilled model, Self Forcing, the method hurts (MUSIQ 69.3 → 47.5).. ◦ (d) “No inference overhead” is relative to the 20-step CFG base only. That base runs at ~4 fps (Table 5), below 16-fps playback, whereas the few-step baselines target real time. This narrows the practical niche relative to the intro’s “deployed checkpoints” framing. ◦ (e) Closely related work is missing. Stable Video Infinity (Li et al., ICLR 2026) also LoRA-tunes a video DiT on its own recycled errors with small data to resist long-horizon drift, and should be discussed and ideally compared.

Questions for authors
  1. Please add lagged identity, flicker/t-LPIPS, anchoring, and cuts for all rows of Table 1, plus a long-range subject-identity measure (e.g., DINO similarity of the main subject to its first appearance). Why does the +real variant have lower lagged identity than the uncorrected base? Is the subject re-sampled at chunk boundaries? Would you restate the headline claims for the zero-real variant?.
  2. Is a high value of α∗\alpha^{\ast} specific to drift? Please report α∗\alpha^{\ast} when Δ\Delta is computed between two clean histories (e.g., two independent single-shot references for the same prompt) and between two independently drifted histories. If these are also approximately 0.95, α∗\alpha^{\ast} reflects how strongly history conditions the velocity, not a property of drift. Also, the corrector sees (zt,hgen,t)(z_t, h_{gen}, t) but not hcleanh_{clean}, so seed-consistency does not imply predictability from its inputs. Validation R2R^2 is 0.39–0.85, well below 0.95. How do you reconcile this with "approximately 95% systematic, therefore learnable"?
  3. Results for LongLive and Rolling Forcing under the 128-prompt, 50-s protocol, with wall-clock/fps next to the few-step baselines, and a discussion of Stable Video Infinity..
  4. An ablation on the deployed (adapted) base under the 128-prompt protocol with SEM: one-step / +DAgger / +contraction. Table 2 is mostly dashes and uses the unadapted base on 5 clips. On the adapted base, Table 10 shows closed-loop increasing saturation drift relative to one-step (0.061 → 0.082). Is the “+4.77 → +1.65” comparison matched in data, prompts, and seeds?.
  5. For the Causal-Forcing teacher under its recommended sampler and cache settings, what is the full 128-prompt drift with and without the corrector?.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.