acceptodds
Under review as a conference paper at ICLR 2027

TGuide: Faster Autoregressive Video Diffusion via Historical Trajectory Guidance

Abstract

Can autoregressive video diffusion be accelerated further when only four denoising steps remain? Existing feature caching approaches struggle in this regime, as large feature changes between denoising timesteps undermine both direct reuse and lightweight predictors. We observe that features across video chunks remain strongly correlated at matching denoising timesteps, offering a complementary source of guidance. Building on this insight, we introduce TGuide, a framework that accelerates autoregressive video diffusion through a lightweight Anchor-Transport-Correct (ATC) predictor guided by historical sampling trajectories. Our ATC combines two complementary references: the current chunk’s preceding-step features preserve its spatial structure, while the previous chunk’s same-timestep features provide information at the target denoising stage. By integrating these references through spatial retrieval and adaptive correction, ATC improves velocity prediction accuracy despite the large gaps between denoising steps. To mitigate error accumulation across both denoising steps and video chunks, we train the predictor on its own autoregressive rollouts, allowing it to adapt to states shaped by its own earlier predictions. An optional confidence-aware scheduler further adjusts the quality–efficiency trade-off. Experiments on Self Forcing, Causal Forcing, and HY-WorldPlay show that TGuide preserves generation quality while achieving denoising speedups of up to over the original 4-step generators. On Self Forcing, TGuide generalizes from 81-frame training videos to 333-frame generation, approximately four times the training length, while exceeding the original model's VBench score.

Primary area:
generative models
Keywords:
Few-Step Video Diffusion, Autoregressive Video Generation

Reviews

Among:
Shape:
1234this paper, Top 17%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.75 over 4 reviews, above 83% of the others.
Reviewer_DNB8OpenReview ↗
Rating
2(−1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Historical Trajectory Guidance (TGuide), a feature-prediction framework for further accelerating few-step autoregressive video diffusion models. The paper observes that four-step video diffusion models obtained through step distillation exhibit substantial feature changes between adjacent denoising timesteps, which makes direct feature reuse and lightweight prediction prone to large errors. In contrast, features from adjacent video chunks remain strongly correlated at matching denoising timesteps. Based on this observation, TGuide introduces two complementary ways of exploiting features: it uses features from the preceding denoising step of the current video chunk to preserve the spatial structure of the current content, while retrieving spatially corresponding information from the previous video chunk at the same denoising timestep. To mitigate the accumulation of prediction errors across both denoising steps and video chunks, the authors further employ self-rollout on-policy distillation, training the predictor on states induced by its own historical predictions.

Critical strengths
  1. The paper is well motivated, with a clear correspondence between its central observation and method design. The paper first analyzes the decline in cross-timestep feature correlation in four-step distilled models and then cleverly exploits the feature correlation between adjacent video chunks at matching denoising timesteps to achieve acceleration. The three components of ATC respectively address the preservation of current content, spatial correspondence across video chunks, and the reliability of historical information, making the overall design intuitive and easy to understand.

  2. The experimental evaluation is relatively comprehensive and specifically examines error accumulation in long-video generation. The paper evaluates the proposed method on three autoregressive video generation models and across both text-to-video generation and camera-controlled image-to-video generation. It reports a broad set of metrics, including VBench, PSNR, SSIM, LPIPS, FLOPs, and measured latency.

Critical weaknesses
  1. The experiments lack direct comparisons with the most relevant cross-chunk caching methods. The main innovation of TGuide is its use of historical features from the previous video chunk at the same denoising timestep. However, the paper does not empirically compare TGuide with recent methods such as C3ache and X-Cache, which also exploit historical information across video chunks. Moreover, the DisCa results in the tables are based on the authors’ own reimplementation rather than an official implementation, making it difficult to reliably judge the relative performance of these methods. As a result, it remains unclear how much TGuide actually improves over existing cross-chunk caching approaches, and the claimed acceleration advantage is therefore difficult to assess conclusively.

  2. The quality improvement is relatively small. This is a minor issue rather than my primary concern, but I would still appreciate further clarification from the authors. In the most important 333-frame Self Forcing experiment, TGuide achieves a VBench Total Score of 82.71, compared with 82.55 for the original four-step model, corresponding to an improvement of only 0.16 points. In the 81-frame setting, TGuide scores 84.23, which is slightly lower than the original model’s 84.28. It is unclear whether the reduction in full denoising computation introduces a loss in generation quality. If the acceleration is achieved only at the cost of a meaningful quality degradation, the practical value of reducing the denoising computation would be limited.

Questions for authors
  1. Could the authors provide direct comparisons with C3ache, X-Cache, or stronger cross-chunk direct-reuse baselines?

  2. Historical features may become unreliable in the presence of rapid motion, abrupt scene transitions, large viewpoint changes, or newly appearing objects. Could the authors report results grouped by motion magnitude or degree of scene change, and analyze whether ATC’s token-wise gate can identify and suppress unreliable historical information in these failure cases?

Reviewer_ugA5OpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes TGuide to accelerate autoregressive video diffusion models that already use four denoising steps. Its lightweight Anchor-Transport-Correct (ATC) predictor combines the current chunk's preceding-step features with the previous chunk's features at the target denoising timestep, using cross-attention retrieval and token-wise gated correction. The predictor is trained with offline regression followed by self-rollout training; a confidence-aware scheduler is optional. Experiments on Self Forcing, Causal Forcing, and HY-WorldPlay demonstrate favorable quality-denoising-cost trade-offs, including a 1.89x denoising speedup on 333-frame Self Forcing generation.

Critical strengths
  1. Further accelerating an already four-step generator is practically relevant. The distinction between cross-timestep and cross-chunk feature correlations provides an understandable motivation for historical guidance (Fig. 2).

  2. The method is clearly described and lightweight. On Self Forcing, ATC uses one DiT block and costs 5.3% of a full DiT evaluation's FLOPs and 6.8% of its latency (Sec. 3.3). The architecture and self-rollout ablations provide evidence that both historical guidance and training on student-generated states contribute to performance.

  3. Evaluation covers three generators and both text-to-video and camera-conditioned generation. The 81-to-333-frame extrapolation on Self Forcing is a useful result: TGuide achieves 82.71 versus 82.55 for the original four-step model while reducing denoising latency from 32.93 to 17.46 seconds (Table 1).

Critical weaknesses
  1. The technical advance over existing prediction and cross-chunk caching methods is not sufficiently established. The paper does identify a challenge: cross-timestep prediction becomes harder under four-step sampling, while motion complicates cross-chunk reuse. My concern is therefore not the absence of a challenge, but whether ATC establishes a sufficiently substantial advance beyond combining a lightweight predictor, historical cross-attention, gated correction, and self-rollout training. In Table 3, adding historical retrieval and correction to the history-free predictor improves the score from 83.86 to 84.08; the correction module itself adds 0.03 over Anchor + Transport. These improvements are encouraging, but no uncertainty estimates establish their reliability. More importantly, the paper cites closely related cross-chunk methods, X-Cache and C3ache, without an empirical comparison. Appendix A.4 explains that their official implementations were unavailable at experiment time, which is a reasonable practical constraint, but the central advantage of learned retrieval and correction over cross-chunk reuse remains incompletely demonstrated.

  2. Preservation of action responsiveness is not demonstrated on HY-WorldPlay. Historical guidance could bias the current prediction toward the preceding view when an action causes substantial translation, rotation, or an abrupt change of direction. ATC is not simply copying features: it uses retrieval and gating, and the first step of each chunk is fully evaluated. Nevertheless, these mechanisms do not by themselves establish that control fidelity is preserved, especially when newly visible regions have little correspondence with the previous chunk. Table 2 reports a five-dimension VBench subset, similarity to the original model, and cyclic-view self-consistency; it does not measure agreement between the generated camera trajectory and the commanded actions. A model that under-executes camera movement could remain visually consistent, so these metrics do not rule out the failure mode. WorldPlay's own evaluation includes rotation and translation errors, and WorldCompass directly evaluates action-following accuracy. Analogous control measurements are needed for the accelerated model.

  3. The evaluation does not adequately test the motion regimes that motivate Transport. The paper does include camera motion, but HY-WorldPlay is trained and evaluated using the same four trajectory families: forward/backward, left/right translation, up/down tilt, and left/right pan (Appendices A.2 and D.3). Held-out images and prompts do not establish generalization to unseen control patterns. The manuscript does not specify the motion magnitudes or provide results stratified by displacement, rotation, view overlap, or newly visible content. Fig. 4 shows a short WorldPlay example, while the additional examples in Fig. 5 are all from Self Forcing; these do not resolve the concern about rapid, compositional, or abrupt camera control. Furthermore, the motivating correlation analysis in Fig. 2 is conducted on Self Forcing and Causal Forcing, not on action-conditioned WorldPlay. I would like evidence identifying when historical guidance helps or becomes unreliable under demanding control inputs.

  4. Long-horizon evidence on the interactive backbone is missing. HY-WorldPlay is evaluated only at 125 frames, approximately 5.2 seconds at 24 FPS (Table 2 and Appendix A.2), which is also its predictor training length. The 333-frame extrapolation result belongs to Self Forcing and cannot establish long-horizon control fidelity on WorldPlay. This matters because approximation errors may affect later actions and revisit consistency over successive chunks. WorldCompass (ICML 2026, Sec. 4.2 and Table 1) evaluates basic and compositional actions at 125, 253, and 381 frames, providing a relevant protocol for testing this concern. I would like a length-dependent comparison of the same WorldPlay checkpoint with and without TGuide. WorldCompass is an RL post-training method rather than a caching accelerator, so it should be used as a control-quality reference or an additional checkpoint evaluated before and after acceleration, not as an unmatched speed baseline. For practical interactive relevance, the reported denoising-only speedups should also be distinguished from end-to-end per-chunk latency: Appendix A.3 excludes KV-cache refresh, VAE decoding, and text encoding.

Questions for authors

correction over an anchor-only predictor and a same-timestep cross-chunk reuse baseline under matched compute and training conditions? Please clarify which component or interaction constitutes the main advance, and report uncertainty for the small architectural gains.

  1. On HY-WorldPlay, does TGuide preserve rotation/translation accuracy and action-switch responsiveness relative to the same unaccelerated checkpoint? Please report control metrics alongside visual quality, particularly for large camera movements and newly revealed regions.

  2. Can you evaluate unseen compositional trajectories, rapid rotations/translations, and abrupt action reversals, with results broken down by motion magnitude or view overlap? An ATC-versus-anchor-only comparison would help determine whether historical guidance remains beneficial in these cases; if dynamic fallback is used, its actual speedup should also be reported.

  3. Can you extend the WorldPlay evaluation to 253 and 381 frames, or comparable longer horizons, using matched conditions and reporting control fidelity, revisit consistency, and latency over time? Please specify the base checkpoint's RL/distillation status and, if feasible, evaluate the same WorldCompass-enhanced checkpoint before and after applying TGuide.

2 comments

I appreciate the authors’ response and their thoughtful answers to my questions. However, the idea of leveraging cross-chunk information for acceleration has been explored in prior work, and the improvements over X-Cache and C3ache appear relatively limited. Regarding Q2 and Q3, I would still like to see visual comparisons of the generated videos. In particular, I remain uncertain whether action fidelity, especially for large-scale motions, can be preserved under acceleration. Therefore, I would like to maintain my original score.

It addresses my concern. I raise my rating to Weak Accept.

Reviewer_k5CUOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes TGuide to accelerate four-step autoregressive video diffusion. Its Anchor-Transport-Correct predictor combines preceding-step features from the current chunk with matching-timestep features from the previous chunk through spatial retrieval and gated correction. The predictor is trained on teacher trajectories and then fine-tuned on its own rollouts, with optional confidence-aware scheduling. Experiments on three video generators report up to a 1.89x denoising speedup with competitive quality, including generalization from 81-frame training to 333-frame inference on Self Forcing.

Critical strengths
  1. The authors analyze the limitations of feature caching in four-step generation and identify preserved cross-chunk correlations at matching denoising timesteps. They exploit this complementary information to improve feature prediction when adjacent-step features become less similar.
  2. The Anchor-Transport-Correct design, together with self-rollout training to reduce error accumulation, is straightforward and effective.
  3. The quantitative and qualitative results across three video generators are encouraging, and the ablation studies support the key designs. Generalization from 81-frame training to 333-frame inference further demonstrates the method's potential.
Critical weaknesses
  1. Please further separate the benefit of historical guidance from that of additional training. DisCa uses only offline regression, while TGuide also receives Stage 2 training with a 14B teacher and a trainable critic. Can DisCa receive the same Stage 2 training?
  2. Experimental verification for several key claims is still insufficient: a) How does ATC compare with directly reusing the previous chunk's features at the same timestep? The FRRF baseline only tests reuse within the current chunk. b) Does historical guidance remain effective under large motion or newly revealed content? HY-WorldPlay uses the same four camera trajectory types during training and evaluation, although the images and prompts are disjoint.
  3. Some improvements are small. The correction module improves the score from 84.05 to 84.08 in Table 3. At 333 frames, the original two-step model is faster than TGuide and only 0.36 points lower. Such marginal improvements could potentially be achieved simply by using different random seeds.
  4. Why does the scheduler use relative hidden-state error rather than velocity error? Please verify whether this measure correlates with generation quality.
Questions for authors

To address the concerns above, further empirical evidence and theoretical justifications are expected.

1 comment

My concerns are partially resolved, and I will maintain the original score.

Reviewer_pWsXOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

TGuide accelerates few-step autoregressive video diffusion by replacing selected full DiT evaluations with a lightweight predictor. Its Anchor, Transport and Correct module combines an earlier denoising feature from the current chunk with a feature from the same denoising step in the previous chunk. The predictor is trained first on offline teacher trajectories and then on its own rollouts. Experiments cover Self Forcing, Causal Forcing, and HY-WorldPlay. On 333-frame Self Forcing generation, TGuide achieves a 1.89× denoising speedup with a VBench score slightly above the original four-step model.

Critical strengths

A Well-Motivated Use of Historical Features. The motivation makes sense. The current chunk provides relevant spatial content, while the previous chunk provides a reference at the same noise level. Attention-based retrieval is a reasonable way to combine these sources when objects move between chunks.

Promising Results and Useful Ablations. The method reduces denoising time across three backbones while largely preserving the reported quality scores. The results on 333-frame videos are encouraging, given that the Self Forcing predictor is trained on 81-frame sequences. The ablations help explain the method. Table 3 shows a benefit from historical feature fusion under matched offline training, and Table 4 shows that rollout training helps more on longer videos.

Critical weaknesses

The Closest Learned Baseline Needs the Same Second-Stage Training.** Appendix A matches the adapted DisCa baseline to ATC in predictor capacity and offline training conditions, but does not say whether DisCa also receives the second-stage rollout training. For Self Forcing and Causal Forcing, this stage uses a 14B teacher and a trainable 1.3B critic. It accounts for a sizeable part of the final improvement: at 333 frames, TGuide's score rises from 82.29 to 82.71 after this stage, while DisCa scores 82.12. Table 3 provides evidence that historical fusion helps after offline training. I would like to see whether that advantage remains when both predictors are trained on their own rollouts. This comparison matters because it would tell us how much the final gain comes from the proposed feature fusion and how much comes from the training procedure.

The Practical Gains Need a Fuller Account of Runtime and Training Cost.** Direct two-step sampling is already a competitive baseline. On 333-frame Self Forcing generation, TGuide improves VBench from 82.35 to 82.71, while denoising time increases from 16.53 to 17.46 seconds. This is a useful quality improvement, but adopting the method also requires training a predictor. The reported latency excludes KV-cache refresh, VAE decoding, and text encoding. The paper also does not report total training GPU-hours. These costs matter when deciding whether to use TGuide for a new backbone, particularly since the second stage involves a large teacher and repeated critic updates. Reporting full generation time alongside training cost would make the practical tradeoff much clearer.

Questions for authors

Q1. How do the adapted DisCa baseline and the predictor without historical feature fusion perform with the same second-stage objective, teacher, and training budget as TGuide? The 333-frame setting would be especially useful here.

Q2. What speedup remains when KV-cache refresh and decoding are included? How many GPU-hours do the two training stages require, and roughly how much generation would recover that cost through inference savings?

2 comments

Dear Authors,

Thank you for the detailed response and for running the additional experiments. The matched second-stage comparison with DisCa addresses my main concern. The five-seed results show that ATC retains a modest VBench advantage at both video lengths under the same training conditions.

The end-to-end timings and training cost estimates also clarify the practical tradeoff. TGuide improves VBench over direct two-step sampling with a small increase in latency. I appreciate the effort to quantify when the training cost can be recovered relative to four-step sampling.

Based on the new evidence, I am raising my rating from 2 to 3 (Weak acceptance). My confidence remains 3.

AuthorsOpenReview ↗

Response to Reviewer pWsX

Thank you for taking the time to review the additional experiments and for raising your rating. We’re glad the matched Stage 2 comparison with DisCa addressed your main concern. Your feedback also helped us present the end-to-end latency and training costs more clearly.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.