acceptodds
Under review as a conference paper at ICLR 2027

Cache Aligned Distillation and Drift Correction for Autoregressive Video World Models

Abstract

Few-step autoregressive video world models must remain responsive to actions while repeatedly conditioning on their own generated history. Few-step distillation enables interactivity in generated worlds, but its errors can accumulate as lost motion, drifting latent statistics, and damaged scene structure. Recent work has improved teacher causality and reduced the training–inference mismatch; yet the ODE stage has not trained a causal student to use its KV cache directly when learning from a causal teacher's rollouts. Our ODE stage trains a four-step student through its inference-time KV-cache path against recorded teacher rollouts, using spatially local Gaussian KL supervision in place of pointwise regression. Autoregressive models are also commonly trained with token normalisation, channel normalisation, and spatial normalisation; the lack of temporal normalisation, however, causes latent values to explode or vanish as rollouts progress. We therefore introduce temporal normalisation to constrain latent statistics against an evolving real-scene reference. Finally, we observe causal autoregressive degradation in the form of highly structured, non-Gaussian, data-dependent noise, and we introduce an online Causal AutoRegressive Noise (CARN) network to model and correct this degradation. On 32 held-out contexts and nine distinct actions, we evaluate 30-second continuations against five open interactive world models. At six seconds, 70.8% of our rollouts remain clean across all six evaluation axes, compared with 24.7% on average externally; relocation is 6.3% versus 55.6%, while directional-control failure remains comparable at 21.1% versus 20.7%. Across longer rollouts, no model dominates every axis: our model preserves actions, high-frequency detail and comparatively low relocation, but accumulates style degradation and geometry drift rather than modes that are harder to recover from such as early collapse or relocation.

Primary area:
generative models
Keywords:
WorldModels, Distillation, AutoRegressive, Forcing

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_CoXtOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper suffers from severe presentation issues, particularly in the Abstract and Introduction, which lack a coherent narrative flow. It is impossible to get the core research problem and understand the proposed method.

Moreover, the writing throughout the paper is obscure and difficult to follow. For instance, the statement: "Standard token and channel normalisation acts within each denoising call, but does not control how the emitted latent distribution evolves across an autoregressive rollout" is difficult to understand.

Furthermore, the main text completely lacks visual illustrations of the method pipeline and qualitative comparisons. Consequently, Consequently, it is exceedingly difficult for readers to grasp the core intuition behind the method, let alone its concrete implementation.

Given that the paper fundamentally falls short of the clarity and quality thresholds expected at ICLR, I recommend rejecting this paper. Additionally, such submissions generated by LLMs should be strongly discouraged and punished by the program committee.

Critical strengths

This paper was most likely written by AI. It is of very poor quality and contains hallucinated content.

Critical weaknesses

The paper suffers from severe presentation issues, particularly in the Abstract and Introduction, which lack a coherent narrative flow. It is impossible to get the core research problem and understand the proposed method.

Moreover, the writing throughout the paper is obscure and difficult to follow. For instance, the statement: "Standard token and channel normalisation acts within each denoising call, but does not control how the emitted latent distribution evolves across an autoregressive rollout" is difficult to understand.

Furthermore, the main text completely lacks visual illustrations of the method pipeline and qualitative comparisons. Consequently, Consequently, it is exceedingly difficult for readers to grasp the core intuition behind the method, let alone its concrete implementation.

Questions for authors

NA

Reviewer_DKQuOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper distills a many-step causal video world model into a four-step autoregressive student for action-conditioned generation, and adds three components to limit error accumulation in long rollouts. The ODE stage trains the student along its own KV-cache inference path on recorded teacher rollouts, with a spatially local Gaussian KL loss instead of pointwise MSE. Temporal Normalisation is a DMD-stage loss that keeps latent statistics close to an evolving real-video reference. CARN is an online residual network that corrects latents before they are written to the KV cache. The model is compared with five public interactive world models on 30-second rollouts from 32 held-out contexts and nine actions, scored on six binary failure axes. At six seconds it has more failure-free rollouts and less relocation (the scene turning into a different place) than the five other models on average, with similar control. At 30 seconds it fails more often than their average on style and geometry, but less often on control, high-frequency detail and relocation.

Critical strengths
  1. The ODE-stage comparison shows a large effect. With the same budget and inference path (Fig. 1), training on the teacher's own cache-aligned rollouts with the local Gaussian KL leaves 51.4% of rollouts clean at six seconds, against 8.7% with a teacher-forced clean prefix and 3.5% with pointwise MSE. Switching to the clean prefix also raises control failure from 3.9% to 43.4%. The study does not include the teacher or the undistilled student, though.

  2. The evaluation re-runs five public world models on the same contexts and actions and scores all of them on six failure axes up to 30 seconds. It also shows results that go against the model: six-second control is only on par with the average of the other five models (21.1% vs. 20.7%), and style and geometry failures at 30 seconds are higher (79.2% and 95.8% vs. 69.2% and 68.9%). My reservation is that the appearance scorers come from prior work (Action Forcing) and were validated against human labels on other videos (Table 1). On these rollouts, only a few cases were checked by hand: 39 of the 411 long-horizon relocation candidates, those that passed an automatic filter, and the candidates for conjuration (objects appearing from nowhere).

Critical weaknesses
  1. The abstract's six-second advantage depends heavily on one automatic relocation test, whose reference set includes each system's real conditioning frames (App. D). This model sees frames 0 to 32 and minWM 4 to 32 (App. B), and both have 6.3% relocation. The other four, with only frame 32, have 44.8% to 87.8%. Conditioning is not the only factor (two ablations of this model relocate in 55.6% and 82.6%), but the comparison leaves it uncontrolled. A long-horizon check found no new relocations, so every later relocation rate comes from this test. Excluding relocation, the Sec. 5.3 claim of the lowest sum of failure rates "at every endpoint" does not hold at 30 seconds (192.8 vs. 188.0 for LingBot). Against minWM, the six-second gap comes mostly from control (21.1% vs. 42.2%), scored in the teacher's action basis with no stated command mapping for other systems. So I consider the comparison in the abstract overstated.

  2. The three contributions are only partly isolated. In the 2×2 ODE study, all three alternatives are heavily degraded, and the study stops before DMD. DMD then changes six-second control failure from 3.9% to 21.1%. Without DMD results from other initialisations, it remains unclear whether the initialisation still matters in the final model. The introduction's CARN numbers come from removing CARN's cache correction, an arm App. F neither plots nor fully specifies. The plotted arm, which drops CARN entirely, also raises control failure to 43.8%, but relocation rises only to 10.4%. Temporal Normalisation has only partial ablations, and the claim that removing it "results in collapse" (p. 2) has no number or figure. There is no table of per-axis failure rates, only stacked-area charts and scattered numbers in the text.

Questions for authors

Q1. If relocation is re-scored for all six systems with frame 32 as the only real frame in the reference set, what are each system's six-second relocation rate and fraction of rollouts clean on all six axes? You checked by hand the 39 long-horizon candidates that passed the automatic filter. How many of the rollouts flagged at six seconds would a human confirm as a real change of scene?

Q2. What do the teacher and the undistilled four-step student score on the same panel? What are the final results when the same DMD schedule starts from the other ODE initialisations?

Reviewer_MThfOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The paper distills a causal action-conditioned world model (Action Forcing, Wan2.1-1.3B) into a 4-step student. It proposes (1) ODE distillation on the teacher’s own rollouts through the KV cache, with a regional mean/variance KL loss instead of MSE; (2) "Temporal Normalisation", a loss that matches latent statistics to the real video during DMD; and (3) CARN, a small network that corrects latents before they enter the KV cache. Evaluation uses six binary failure detectors on 32 contexts x 9 actions, against five open world models.

Critical strengths
  • Training the ODE stage the same way the student runs at inference is a reasonable idea.
  • The 2×2 ODE ablation (context x loss) is clean and matched.
Critical weaknesses
  • There are no videos or supplementary material for a video paper. The main paper has no qualitative results, and the appendix has only small frame strips up to 14 s. I cannot verify any claim about 30-second rollouts.
  • All results come from custom detectors taken from an unpublished preprint (Action Forcing). Their validation is weak (AUC 0.68–0.78). There are no standard metrics (FVD, VBench) and no user study.
  • The external comparison is not fair. The proposed model gets 33 context frames, but four baselines get only 1 (Appendix B). This mostly explains the headline “6.3% vs 55.6% relocation”.
Questions for authors
  • Can you share full videos for all models?
  • What are the results when all models get the same context length?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.