acceptodds
Under review as a conference paper at ICLR 2027

How to Train Your Video World Model for Long-Horizon Rollout

Abstract

Video world models must sustain autoregressive generation long after their initial observations leave the attention window. How should they be trained for this regime? Recent work advocates conditioning on clean history during pre-training to match inference, but the pretrained model is often only the starting point for further training. We study how this choice affects the complete pipeline for action conditioned world models without a separate bidirectional teacher. In this setting, the pretrained causal model provides both the student initialisation and the teacher for post-training. On Minecraft, we compare Diffusion Forcing, clean prefix teacher forcing, and chunked teacher forcing at equal pre-training compute. Each model undergoes flow map finetuning followed by Self Forcing against its own teacher, with evaluation over 1000 generated frames. The recipe rankings change after post-training. Diffusion Forcing initially performs worst beyond the attention window, but subsequently outperforms both alternatives through frame 400 and matches chunked teacher forcing at longer horizons. Its Fréchet Video Distance over frames 401–1000 falls from 329 to 86, compared with 132 to 91 for chunked teacher forcing. Clean prefix teacher forcing is best at neither stage. The post-training stages play complementary roles: qualitative rollouts suggest that flow map finetuning increases dynamic content and Self Forcing improves its persistence. These results show that alignment with inference during pre-training alone does not determine rollout quality. Training recipes should be evaluated through the full pipeline and at long horizons.

Primary area:
generative models
Keywords:
world models, video generation, autoregressive generation, flow matching, diffusion forcing, exposure bias, long-horizon rollout

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_tmWsOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper studies video pretrained-model causal fine-tuning for world modelling (e.g. autoregressive generation) for long-horizon tasks. The author explores different prior approaches and offers some insight into which one works best for longer generation on the Minecraft benchmark.

Critical strengths
  • Long horizon generation is an important problem, and exploring existing techniques to know which ones work best is useful to be able to develop new ideas and offers practical insight.
Critical weaknesses

This paper reads mostly like a class project with little to no novelty. The paper writing is also not good enough; multiple non-standard issues. Background method, experiments details are intertwined in places. There's a very underdeveloped related work section. The experiments section lacks a standard structure for baseline, dataset, etc. description. Overall, the writing lacks clarity. The method itself is limited and only explores existing ideas, experiments with some of them and offers little novel contribution to the area. I think it's just an early report of exploration, and not a proper thesis of what the contribution/finding is. Of course, not yet ready for publication!

Questions for authors

None

3 comments

Thank you for your review. We agree with your point that knowing which existing techniques work best "offers practical insight" and that is the aim of the paper. We also agree that the presentation can be easier to follow. In the submission, much of the information you ask for was in the appendix, and the main text did not point to it clearly enough. The revision fixes this:

  1. Contributions are stated as findings (F1-F4, Sec. 1), so the paper's thesis is explicit: rankings after pre-training do not predict rankings after post-training, and we explain why.
  2. The experiments open with a standard setup (Sec. 4), organised as Data and recipes, Evaluation and Metrics. Each points to its full description, which was already in the appendix:
    • recipes, attention masks and compute matching: App. A
    • data and tokeniser: App. B.1
    • inference and evaluation protocol: App. B.4-B.5
    • all metrics: App. C.6-C.7
  3. Related work. Sec. 2.6 covers causal video post-training (CausVid, Self Forcing, Self-Forcing++, Causal Forcing and AAPT). Sec. 2.3 and App. D cover exposure bias and history guidance, few-step generation, and interactive world models with memory (DIAMOND, Genie, Oasis, MineWorld).
  4. New evidence, shared with the other reviewers:
    • frame quality, action following and dynamic content (Table 2)
    • a crossed teacher/student study, and DF re-trained with the other recipes flow-time prior (Fig. 5, App. C.2)
    • rollouts to 5,900 frames (about five minutes) and at 352×640 (App. C.8).

The review does not name a specific soundness concern. Could you point to the claims or experiments you find unsound, and tell us what evidence would change your assessment? We will address them during the discussion.

I thank the authors for the rebuttal. It is a challenging time for conference reviewing, and I want the authors to know that I acknowledge their efforts. I looked at the revised draft and the responses to my review and those of other reviewers. Let me reiterate that my main criticism is that this paper is clearly underdeveloped and lacks a clear thesis. Surely, there are some experiments along different axes of known approaches, which were further improved through the rebuttal responses and additional experiments addressing other reviewers' comments. However, I still do not see any clear findings or interesting insights that are significant. I read the document, and yes, there are different findings from trying out different things, but they do not form a cohesive contribution.

I'd like the authors to read their revised contributions themselves.

F1 A controlled comparison of the full pre-training and post-training pipeline at equal
pre-training compute. Evaluation over 1000 frame rollouts shows that recipe rankings
change after post-training.

F2 Evidence from predictions conditioned on ground truth and history noise experiments that
a mismatch between training and inference contributes to DF’s initial rollout deficit.

F3 An analysis of the distinct effects of post-training. Flow map finetuning increases dynamic
content across the three recipes, while Self Forcing improves overall rollout quality. The
complete post-training pipeline changes the preferred cache policy in favour of retaining
K/V as the window slides.

F4 Crossing students with teachers shows that the DF teacher stabilises the PF student.
  • These findings are all over the place. Yes, I read about the change in rankings between pre-training and post-training. But that does not help us conclude anything meaningful yet. I think this is a nice early-stage exploration, but it does not establish a single clear finding. It also sounds like someone conducted a lot of experiments and presented a summary of them -- lacking a general or robust understanding. Therefore, my review and score remain the same. The work is not yet ready for publication. The authors should spend more time on the focus and clarity of their contributions.

  • I'm not going into methodological and results details because I think the overall research question/importance/ contribution is lacking a lot. But, since the author wants me to stretch that too. I think it is important to establish this type of phenomenon across multiple datasets and model families, etc., to be able to make claims that the author is suggesting in this work.

  • One is not expected to read the appendix; most of the content is in the main paper, and the pages should be enough to write the paper. So, it's not a good practice to point reviewers to a long list of appendices to follow a paper (appendices can be useful for reproducibility and supplementary material, but not a prerequisite to read and understand a paper, AFAIK). Also, they should be written in a standard style of writing that we know of for machine learning papers. What I am trying to suggest is that it's hard for me to follow this paper! And I know the authors have the details somewhere in the appendix, but that is not an effort for the reader to be able to follow the work!

  • Also, on a side note, when did we start using the term "recipe" to describe settings and methods in machine learning? As I mentioned last time, the paper has a very unusual writing style. Also, we shouldn't take writing lightly if we think we're writing these papers for humans to read. Perhaps if it is written for AI, then it's up to AIs. But as a human, I'll keep my bargain: the writing and contribution of the work need to be much, much better for conference publication.

Thank you for taking another look at the paper. The success of large language models shows that post-training can substantially change a pretrained model. For autoregressive video world models, this raises the question: how do pretraining and post-training choices work together to shape long-horizon generation? We study how the pretraining objective affects the final model, what each post-training stage contributes, and how teacher choice affects the outcome.

Our main finding is that the pretraining objective that performs best before post-training does not necessarily produce the best model after post-training. In our experiments, Diffusion Forcing (DF) starts with the weakest long-horizon rollouts but performs best or comparably after the full pipeline. We also find that flow map fine-tuning and Self Forcing have distinct effects, and that teacher choice affects the final results.

These experiments address connected parts of the same question. The pretraining comparison shows that post-training can change the ranking of the models. The stage evaluations and teacher comparison help explain how the post-training process affects the final results.

Prior work commonly pretrains with prefix forcing and then post-trains with a DF teacher. Interestingly, our experiments show that pretraining with DF and post-training with a DF teacher performs better. When we fix the prefix forcing student, using a DF teacher improves mean FVD from 135 to 98.

We hope this clarifies our contribution: a controlled empirical study of how pretraining, post-training stages, and teacher choice interact in long-horizon generation for video world models.

Reviewer_DtLWOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper investigates how different training recipes, encompassing causal pre-training and post-training, affect long-horizon autoregressive video generation without an additional bidirectional teacher. Using the same action-conditioned Minecraft video world model, the authors compare three complete training recipes based on Diffusion Forcing (DF), clean-prefix Teacher Forcing (PF), and Chunked Teacher Forcing (CHUNK-4), with approximately matched pre-training FLOPs.

Critical strengths
  1. The paper provides a valuable empirical comparison of complete training recipes, covering both causal pre-training and the corresponding post-training choices. The results offer useful guidance for the video world modeling community (Table 1).
Critical weaknesses
  1. The source of DF's large post-training gain is unclear. DF uses a uniform flow-time prior (Sec. 3.1) and a different history configuration in Stage 1 (Sec. 3.2). Stage 2 also changes the student initialization and causal teacher together (Sec. 3.3). These coupled changes make it difficult to attribute the gain to any single design choice. A small controlled ablation would help.
  2. The evaluation relies mainly on FVD computed from 16-frame I3D clips (Appendix B.5), which does not directly measure long-term consistency. Sec. 4.5 shows that lower FVD can coincide with changes in object identity and background appearance. Reporting background consistency, motion smoothness, and temporal flickering would provide complementary quantitative evidence.
Questions for authors
  1. Can the authors isolate the main source of DF's post-training improvement, for example by fixing the student or teacher while controlling the flow-time prior?
  2. Could the authors report complementary metrics such as background consistency, motion smoothness, and temporal flickering on the existing rollouts?
2 comments

Thank you for the careful reading and the two concrete questions.

Q1. Where does DF's post-training gain come from?

In short: Our results suggest that i) the teacher sets stability, and ii) the flow-time prior matters before post-training but not after it.

You list three coupled changes. We take them in turn.

  • Flow-time prior (Sec. 3.1). We pre-trained DF again with the logit-normal prior the other recipes use, changing nothing else, and post-trained it with 3 seeds (App. C.2, Table 9).
    • Before post-training the prior matters: FVD falls from 329 to 217, but DF stays behind PF (177) and Chunk-4 (132). FVD also overstates the gain: the logit-normal model follows actions worse (IDM F1 0.755 vs 0.795).
    • After post-training the difference is not detectable: 101 vs 94 (seed ranges 76-134 and 86-99 and +8 over seeds).
    • Our conclusions therefore hold under either prior. A choice worth 112 FVD before post-training is invisible after it, which supports comparing recipes through the full pipeline.
  • Student and teacher (Sec. 4.3). We fix the student and swap the teacher: each Stage-1 student is post-trained with its own teacher and with the DF teacher, 3 seeds each (Fig. 5, Table 14). The teacher sets stability, as with the DF teacher, the PF student's FVD goes from 135 to 98, and its range over seeds narrows from 98-170 to 95-102.

Q2. Background consistency, motion smoothness, temporal flickering.

We computed all three on the existing rollouts. We provide definitions in App. C.6 and all values are frames 401-1000, read against ground truth decoded through the same VAE.

  • Temporal flickering: reported, adapted (FC, Table 13). VBench's score is the mean absolute difference between consecutive frames, which assumes a static camera. Our camera turns in almost every frame, so that score would measure camera speed. FC i) removes the global camera shift by phase correlation, ii) compares only fine detail (high-pass images), and iii) divides by the detail present, so a blurred rollout cannot score well. After post-training all recipes are at 0.78-0.80, against 0.843 for ground truth.
  • Background consistency: computed, as in VBench (App. C.6). VBench averages the CLIP similarity of each frame to the first frame and to the previous frame. We compute both terms with the same CLIP model. The first-frame term mostly measures how far the player has walked, so it does not test the model. The adjacent-frame term is within 0.003 of ground truth for every pre-trained, post-trained and history-noise model, so it does not separate the recipes. We state this in App. C.6 rather than adding a column of identical values.
  • Motion smoothness: computed, as in VBench (App. C.6, Table 13). VBench drops every other frame, re-synthesises it with its AMT-S interpolator and scores the match. Teacher-forced rollouts match ground truth (0.9745 vs 0.9746), so our tokeniser (Wan VAE) does not bias the metric. AR rollouts score 0.003-0.008 above ground truth so since a static video scores 1, they are slightly too smooth. Post-training shrinks the gap for every recipe (PF +0.008 to +0.003, Chunk-4 +0.007 to +0.004, DF +0.006 to +0.005). All differences are below 1% of the scale, so motion stays plausible and the metric separates recipes only weakly.
  • Dynamic entities and short-term continuity (complementary). We identify animals and other NPCs in sampled frames, then assess their motion and coherent appearance over two-second clips. Successful occurrence increases after post-training for all recipes, and DF's NPC persistence reaches 95.9%, close to ground truth (Table 2, App. C.7). This gives an entity-level check that frame-similarity metrics cannot.

We hope this answers both questions. If it does not, could you tell us what further evidence or analysis would change your assessment? We would be glad to provide it during the discussion.

I thank the authors for their efforts on the rebuttal. My concern about the evaluation metrics remains insufficiently addressed. It is well known that VBench fails to reflect methodological advantages in many evaluations, and I suggest that the authors use additional evaluation criteria, including VLMs, to support the assessment. I have also reviewed the revised draft and the responses to my comments and to those of the other reviewers. I agree with reviewers tmWs and ggEz that the methodological novelty is limited and consists mainly of combining and comparing existing training methods. I therefore keep my original score unchanged.

Reviewer_ggEzOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies the training strategies of the video world model in long-term autoregressive generation. In the Minecraft environment, Diffusion Forcing, clean history frame-by-frame prediction, and block prediction are compared with the same pre-training computing power. They are uniformly fine-tuned through streaming mapping and then trained with Self Forcing. The experiments reveal that Diffusion Forcing, which performed the worst in long-term generation during the pre-training stage, achieves the best or comparable performance after the training, indicating that consistency between pre-training and inference conditions does not guarantee the optimal final result. It is necessary to evaluate based on the complete training process and long-term generation. The main contribution of this paper is empirical analysis and training experience, rather than proposing a new algorithm.

Critical strengths
  1. This article focuses on the accumulation of errors in long-term generation, and the research topic has practical value.

  2. By comparing the complete training process under the same pre-training computing power, the comparison is relatively fair.

  3. Revealing that post-training will change the ranking of methods, providing empirical evidence for the selection of training strategies.

  4. Evaluating up to 1000 frames and combining historical noise and cache experiments to analyze the influencing factors of long-term generation.

Critical weaknesses
  1. The method innovation is limited, mainly consisting of the combination and comparison of existing training methods.

  2. The experiments only cover Minecraft, a single architecture and low resolution, and the generalization is not clear.

  3. It mainly relies on FVD and KL, which cannot fully reflect the quality of long videos, and should supplement indicators such as VBench.

  4. The improvement of dynamic content and its continuity mainly depends on qualitative presentation, lacking quantitative evidence.

  5. This paper studies the action-conditioned world model, but lacks the evaluation of indicators related to actions and downstream planning assessment.

Questions for authors

For the action-condition world model, I am looking forward to seeing metrics related to actions as well as more meaningful metrics like those in VBench.

2 comments

Thank you for recognising the fair comparison and the ranking reversal, and for a summary that captures our central finding.

Action-related metrics. Following MineWorld, we run OpenAI's VPT inverse dynamics model on each generated video and compare the actions it reads with the logged ones. We report F1 over the movement keys and the correlation for the camera. Ground truth reaches F1 0.937, and shuffled actions give 0.08-0.17. Pre-trained DF follows actions worst (F1 0.795). After post-training all recipes are level (0.84-0.86).

VBench-style metrics. We added frame quality (ΔMUSIQ, VBench imaging quality, Table 2) and flicker (FC, VBench temporal flickering adapted to a moving camera, Table 13). The general response explains the adaptations. We also computed VBench background consistency and motion smoothness unchanged. Both stay within 0.01 of ground truth for every model, so they do not separate the recipes (App. C.6).

Dynamic content and continuity. We address the missing quantitative evidence with a two-stage evaluation inspired by T2V-CompBench: identify entities, then assess their motion. NPCs provide a test of infrequent, structured dynamics: their appearance must remain coherent as they move. GPT-6.1 Terra (low reasoning) screens 25,600 frames from 256 shared test episodes across GT and nine recipe/training-stage combinations, then assesses the 1,042 positive samples in two-second clips.

Successful NPC occurrence rises from 0.27% to 2.81% for DF, 0.43% to 1.46% for PF, and 0.78% to 1.97% for Chunk-4 after the full pipeline. Degraded entities do not count as successful. NPC persistence (successful samples divided by successful plus degraded samples) reaches 95.9% for DF, close to ground truth (96.6%). Against human labels, screening reaches 85.7% F1 on 68 frames and temporal verification 79.5% F1 on the 73 clips with conclusive labels (Table 15). Sec. 4.2 and Table 2 present the results, and App. C.7 gives the protocol, prompts, validation (Tables 15-16) and the breakdown by stage and horizon (Table 17, Figs. 14-16).

These measurements support the recovery of dynamic content suggested by forward KL and the qualitative examples. Flow map fine-tuning increases successful occurrence for every recipe. Self Forcing further increases it for PF and Chunk-4. In contrast, DF's occurrence decreases while its NPC persistence rises from 89.7% to 95.9%. The overall rollout-quality improvement remains supported by FVD.

Generalisation (App. C.8). The architecture is held fixed by design: the recipes differ only in their objective and attention mask, so any architectural change would confound the comparison. One complete pipeline costs full pre-training plus two post-training stages, and each is evaluated over 1,000 frames. Within that budget we add:

  • Rollouts to 5,900 frames:
    • The post-trained ordering holds (DF < Chunk-4 < PF on FVD).
    • DF beats Chunk-4 significantly in 3 of 5 1,000-frame bins.
  • The DF pipeline at 352×640 (4× the pixels):
    • FVD over frames 401-1000 is 70.
    • It stays within 90-128 per 1,000-frame bin up to 5,900 frames.
    • This is a single run without high-resolution baselines, so it shows the pipeline transfers, not a ranking.

Planning. We evaluate the model as an action-conditioned simulator, which the IDM tests directly. Planning adds a policy and a task, and is out of scope.

Novelty. We do not propose a new algorithm. As your summary puts it, consistency between pre-training and inference does not guarantee the best final result. This was not known, and it changes how recipes should be chosen. The contribution is findings that change practice (F1-F4, Sec. 1), notably:

  1. The ranking reverses. The recipe that is worst at long horizons before post-training is best or level after it.
  2. The stages and the teacher have distinct effects. Flow map finetuning increases dynamic content across all three recipes, Self Forcing improves overall rollout quality, and a DF teacher stabilises the PF student (Fig. 5).

Neither follows from the individual methods.

We hope these additions address your concerns. If anything remains, we are happy to discuss it and provide it in the discussion period.

Thank you for adding action-related metrics and VBench metrics. These make the evaluation more complete.

However, my main concern remains: the findings are not used to improve the method or training process. Overall, the paper reads more like an engineering study with familiar findings and offers limited new research insights. Given these limitations, I believe the paper is close to but slightly below the acceptance threshold for ICLR.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General response to all reviewers

We thank the reviewers for their feedback. We appreciate ggEz’s recognition of the change in recipe rankings after post-training and DtLW’s assessment that comparing complete training recipes provides useful guidance for video world modelling. We have added evaluation beyond FVD (ggEz, DtLW), controlled ablations (DtLW), longer and higher-resolution rollouts (ggEz), and clearer presentation (tmWs). Changes are marked in colour.

Additions

  1. Metrics (Table 2; Apps. C.6–C.7), with ground-truth references:

    • VBench: imaging quality (ΔMUSIQ relative to ground truth), background consistency, motion smoothness, and temporal flickering adapted to camera motion (FC).
    • Actions: movement-key F1 and camera-motion correlation between logged actions and those inferred by the VPT inverse dynamics model [1], following MineWorld [2].
    • NPCs: frame screening followed by two-second motion and identity verification, inspired by T2V-CompBench [3]. Occurrence is the fraction of evaluable sampled frames containing at least one coherently moving NPC and persistence is the successful fraction among samples classified as successful or degraded. Protocol and human validation are in App. C.7.
  2. Controls: compared each student checkpoint post-trained with their native teacher vs the DF teacher, and DF pre-training with the other recipes’ logit-normal prior, followed by both post-training stages. Each condition uses three Self Forcing seeds (Fig. 5; App. C.2).

  3. Extended evaluation: 5,900-frame rollouts and one DF run at 352×640 (App. C.8).

  4. Presentation: explicit findings F1–F4 and a reorganised experimental setup (Secs. 1, 4).

Findings

Quality and action metrics use frames 401–1000; NPC metrics sample throughout the rollout.

  • Action following corroborates the original finding that DF catches up after post-training. Pre-trained IDM F1 is 0.795 for DF, 0.845 for PF, and 0.843 for Chunk-4. After post-training, scores are similar across recipes (0.842–0.861; ground truth: 0.937).
  • Successful NPC occurrence increases for every recipe: DF 0.27% → 2.81%, PF 0.43% → 1.46%, and Chunk-4 0.78% → 1.97%, versus 8.01% for ground truth. DF has the highest final occurrence and persistence; persistence reaches 95.9% (ground truth: 96.6%).
  • The controls show that teacher choice matters. With the PF student fixed, the DF teacher improves mean FVD from 135 to 98. Changing DF’s flow-time prior improves pre-training FVD from 329 to 217, but yields no detectable improvement after post-training over frames 401–1000 (101 vs 94).
  • Temporal scores differ little in absolute terms: adjacent-frame background similarity stays within 0.003 of ground truth; motion smoothness is 0.003–0.008 above it. Post-trained FC is 0.784–0.795 (ground truth: 0.843).
  • The post-trained FVD ordering holds to 5,900 frames. The higher-resolution run also reaches 5,900 frames, but does not compare recipes at that resolution.

The NPC metrics broadly support the final FVD ordering; ΔMUSIQ and motion smoothness favour PF. These results reinforce our recommendation to evaluate complete pipelines at long horizons using complementary metrics.

References

[1] Baker et al. Video PreTraining (VPT): Learning to Act by Watching Unlabeled Online Videos. NeurIPS, 2022.

[2] Guo et al. MineWorld: a Real-Time and Open-Source Interactive World Model on Minecraft. arXiv, 2025.

[3] Sun et al. T2V-CompBench: A Comprehensive Benchmark for Compositional Text-to-video Generation. CVPR, 2025.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.