acceptodds
Under review as a conference paper at ICLR 2027

ForgeWM: Progressive Causal Training for Few-Step Action-Conditioned Video World Models

Abstract

Action-conditioned video world models require low-latency causal generation and reliable responses to game-native controls. Although causal distillation enables one- or few-step video synthesis, extending it to interactive world models remains challenging, as discrete keyboard states and continuous mouse motion must remain aligned with temporally compressed latent chunks during causal training and autoregressive rollout. We introduce ForgeWM, a progressive framework that transforms a bidirectional action-conditioned video generator into efficient few-step world models through domain adaptation, teacher-forced causal training, causal consistency distillation, and on-policy distribution matching with a bidirectional teacher. The budget-specialized students operate at steady-state denoising budgets of 1, 2, and 4 steps. ForgeWM further supports a dual-path deployment protocol combining latency-critical interaction with optional replay-time refinement, where the one-step student re-noises and refines its saved draft. On paired Minecraft trajectories, ForgeWM leads the evaluated systems in Imaging Quality, reference-aligned motion-profile agreement, action-sign accuracy, and mouse-control accuracy, while achieving the lowest reference LPIPS; the same four-stage recipe transfers to gamepad-controlled FPS gameplay. Replay-time refinement matches four-step reference quality while remaining roughly three times closer to the experienced trajectory than regeneration from noise. These results demonstrate ForgeWM's effectiveness for controllable few-step video generation.

Primary area:
generative models
Keywords:
World Models, Video Generation

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_bYmCOpenReview ↗
Rating
2(+1)
Confidence
2
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents ForgeWM, a progressive training framework for converting a bidirectional action-conditioned video generator into causal few-step world models for real-time interactive generation. The proposed pipeline consists of four stages: bidirectional domain adaptation, teacher-forced causal training, causal consistency distillation, and on-policy distribution matching. Based on this pipeline, the authors obtain budget-specialized students operating with 1, 2, or 4 denoising steps while preserving frame-aligned keyboard, mouse, and gamepad controls. The paper further introduces a dual-path deployment scheme, where low-step models are used for latency-critical online interaction, and the generated trajectory can optionally be improved afterwards using Replay-Time Refinement. Experiments are mainly conducted on Minecraft trajectories, with an additional transfer experiment on gamepad-controlled FPS gameplay.

Critical strengths
  1. The paper presents a practical training pipeline for converting an action-conditioned video generator into 1-step, 2-step, and 4-step interactive world models. The resulting system provides a useful trade-off between generation quality and inference latency.
  2. The paper considers several practical aspects of interactive world-model deployment. It evaluates different inference budgets and control settings and also demonstrates transfer beyond the main Minecraft environment.
Critical weaknesses
  1. The proposed pipeline mainly combines existing techniques, and the paper does not provide enough evidence to justify why this particular combination is necessary or technically non-trivial. I do not think a paper must introduce a fundamentally new method to be valuable. However, when the main contribution is the integration of existing techniques, I expect stronger experiments showing why each component is needed and why simpler alternatives are insufficient. The current ablations mostly show that the proposed stages can be combined into a working pipeline. The stage-wise ablation shows that individual stages affect performance, but it does not sufficiently establish why this particular sequence of existing techniques is necessary, or whether simpler or alternative combinations could achieve similar results. I therefore find the current empirical justification insufficient for establishing the significance of the proposed training recipe.
  2. The experimental comparison is insufficient at both the system level and the distillation-method level. The main baselines are relatively limited, and several more recent interactive world models are not considered. I understand that direct comparison can be difficult because the paper builds on an older base model. However, recent methods are still important for understanding whether the proposed design remains necessary. In particular, Matrix-Game 3.0 demonstrates a strong alternative based on a bidirectional student and self-generated multi-segment training. This makes it important to explain and experimentally justify why a block-wise causal student is preferable to a bidirectional student design in this setting. The paper should ideally compare causal and bidirectional students under the same backbone and data. Since the main contribution is also framed as a distillation pipeline, the paper should include controlled comparisons with related distillation methods such as mentioned AAPT. Without these comparisons, it is difficult to determine whether the gains come from the proposed pipeline itself or simply from applying existing distillation techniques to the chosen base model.
  3. I am not fully convinced that Replay-Time Refinement is sufficiently connected to the main contribution of the paper. The paper shows that this procedure can improve the quality of a saved trajectory while remaining closer to the original rollout than regeneration from noise. This can be useful for replay or video export. However, it does not improve the quality or latency of the actual real-time interaction. It is also largely independent of the causal distillation pipeline highlighted in the paper. I therefore think its role as a major contribution requires stronger motivation. Otherwise, it may be more appropriate to present it as an optional deployment feature rather than a central component of the method.
Questions for authors

See weaknesses above.

Reviewer_jMtfOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces ForgeWM, a multi-stage training method for few-step action-conditioned video world models. Compared with causal distillation for general video generation, the method focuses on how to preserve aligned discrete/continuous game control during training and inference. Its key idea is progressively training the models. Specifically, the authors build a domain-adapted bi-directional teacher and a causal branch. Then, train the causal branch with consistency distillation for few-step generation. The student is trained on its own generated history to reduce errors during rollout. Separate video and action caches help keep the controls aligned with the generated video. The authors also add an optional SDEdit step with the same actions to improve visual quality.

Critical strengths
  1. Practical output. Whole framework combines few-step generation for game controls, making it useful for interactive video generation. The optional replay step also allows users to improve visual quality after generating a quick draft.

  2. Thorough human evaluation. The paper provides a blinded user study result to evaluate control behavior and perceived video quality.

Critical weaknesses
  1. Limited novelty. Causal consistency initialization followed by on-policy distribution matching has already been explored in prior work, e.g., Causal Forcing++. The SDEdit-like replay refinement also a common trick. The main contribution seems to be combining existing components for action-conditioned generation, but it is unclear what new mechanism is introduced beyond this integration.

  2. Complicated training procedure. The 4-stage training involves several training objectives and hyperparameters. This may make reproduction and extension difficult. Comparisons with a simpler training pipeline and an analysis of sensitivity to these settings would help clarify the contributions.

Questions for authors
  1. What is the actual control to output latency, including VAE decoding?

  2. Could the authors clarify what makes action-conditioned generation particularly challenging in the few-step setting? Actual failure examples from a standard few-step distillation would help explain the specific challenges and why the proposed training stages are needed.

Reviewer_6GXiOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

ForgeWM combines domain adaptation, teacher-forced causal training, consistency distillation, and on-policy distribution matching to produce action-conditioned video models with steady-state budgets of 1, 2, and 4 steps. It evaluates these models on Minecraft, demonstrates separately trained FPS-domain adaptation, and introduces replay refinement using the deployed checkpoint.

Critical strengths
  1. Useful quality–speed trade-offs. ForgeWM-2 improves LPIPS over Matrix-Game 2.0 from 0.6443 to 0.6171 while reducing reported chunk sampling time from 370.9 to 239.7 ms (Table 1). The four-step model's 68.8% visual-quality preference share provides complementary perceptual evidence (Figure 4).
  2. Practical replay refinement. The same checkpoint improves reference LPIPS from 0.6532 to 0.6155 while remaining substantially closer to the saved draft than regeneration from noise (Table 2), without training a separate refiner.
Critical weaknesses
  1. The methodological contribution is insufficiently established. The action interface is inherited from Matrix-Game 2.0 (§3.2), while the training sequence closely follows Causal Forcing++ and the adaptation/distillation framework of minWM. The paper does not identify and isolate a specific technical advance required by game-native control. More importantly, Appendix B states that the baselines receive none of ForgeWM's additional training on 40,000 GF-Minecraft clips. Table 1 therefore confounds the proposed recipe with domain adaptation and additional training; it cannot establish the recipe's incremental value.
  2. The central training and interaction claims are not adequately tested. In Table A3, Stage 3 improves IQ from 0.659 to 0.716 but worsens LPIPS from 0.605 to 0.617, with unchanged subject consistency. Its incremental benefit at the crucial one- and two-step budgets is not isolated. Moreover, KCtrl measures net motion sign, and Flow Profile is insensitive to proportional changes in motion magnitude (Appendix B.2); neither establishes precise temporal or magnitude control. Reported latency excludes VAE decoding and initial-chunk cost. These measurements leave the motivating claim of reliable, low-latency interaction insufficiently supported.
Questions for authors
  1. Additional validation: With matched domain data and training budgets, how does ForgeWM compare with a direct Causal Forcing++ adaptation, and what does Stage 3 add at one and two steps?
  2. Clarification and validation: How are incoming actions incorporated within twelve-frame chunks? What are the end-to-end action-response delays, including decoding, on specified hardware, and how accurately does the model follow action switches and changes in control magnitude?

Comments on the paper

6 comments
AuthorsOpenReview ↗

General response (1/2)

We thank the reviewers for their constructive feedback. We are glad that the practical quality–latency trade-offs of ForgeWM and its replay refinement capability were recognized. The reviews also raised important questions about the contribution beyond existing causal distillation methods, the effect of additional in-domain training, and whether our evaluation sufficiently captures the latency and controllability needed for interaction. We have conducted several new experiments to address these points and incorporated the results into the revised manuscript (changes highlighted in blue).

Contribution.

ForgeWM builds on the broader line of autoregressive video distillation methods, including Self-Forcing, Causal Forcing, and Causal Forcing++, rather than introducing a new distillation objective. Our focus is on bringing progressive causal training to interactive world models with game-native controls. Specifically, we combine a domain-adapted bidirectional teacher with causal distillation while maintaining frame-aligned discrete and continuous actions across latent chunks and causal caches. The framework produces budget-specialized 1-, 2-, and 4-step students and also transfers to gamepad-controlled FPS environments, retaining the same backbone, action-module topology, and four-stage training objectives with only an input-level adaptation to the control interface. Replay refinement is presented as an optional deployment feature rather than part of the core training contribution.

To better understand how much each component contributes, we have added controlled comparisons with simpler training alternatives. In E1, we train Matrix-Game 2.0's released distilled student using our data and DMD configuration, and compare against a direct Causal Forcing++ adaptation without our domain-adapted teacher. These experiments use the same 40k clips and matched Stage-3 DMD settings and iterations, allowing us to examine the effect of the preceding training recipe. We also evaluate the Stage-2 checkpoints directly at 1, 2, and 4 steps (E2), and study how removing or shortening the consistency initialization affects subsequent DMD training (E6).

For the interaction-related concerns, we have measured end-to-end latency including decoding and cache updates (E3), tested action reversals both at and within chunk boundaries (E4), and added motion-magnitude-sensitive metrics (E5). We also evaluate the released Matrix-Game 3.0 on our evaluation inputs (E7) to provide a comparison with a more recent interactive world model.

The reference-based comparisons use 200 paired trajectories from Table 1, with an additional set of 200 trajectories from the data_269 subset of GF-Minecraft for cross-model comparisons. We further use 20 independently collected scenes with six constant actions for no-reference quality and control evaluation. The full protocols and results are included in the revised Section 4.4 and Appendix F.

AuthorsOpenReview ↗

General response (2/2)

Main findings. The new experiments help clarify both the benefits and the limitations of our training recipe.

First, with matched data and Stage-3 DMD budgets, ForgeWM converges considerably faster than directly distilling the Matrix-Game 2.0 student. At 400 DMD iterations, LPIPS is 0.561 vs. 0.661 on the paired trajectories of Table 1, and 0.398 vs. 0.462 on the additional set. ForgeWM also maintains higher aesthetic quality across the evaluated budgets. The fidelity gap becomes smaller with longer training: at 4,000 iterations, ForgeWM-4 still has lower LPIPS on the Table 1 trajectories (0.618 vs. 0.652), while the Matrix-Game 2.0 student performs better on the additional set (0.415 vs. 0.477), where ForgeWM produces larger motion than the reference. Compared with the released Matrix-Game 2.0 checkpoint without further training, ForgeWM-4 achieves lower LPIPS on both sets (by 0.030 and 0.040). We also find that the domain-adapted teacher improves LPIPS by 0.043 and Imaging Quality by 0.028 over the direct CF++ adaptation, with an IQ improvement of 0.027 on independent scenes.

Second, the role of Stage 3 becomes much clearer at low denoising budgets. At one step, the consistency-initialized model gradually freezes during autoregressive rollout, with its motion dropping to only 2–3% of the reference near the end of the sequence. This persists even when consistency distillation is extended from 1k to 6k iterations. Stage 3 prevents this collapse, reducing LPIPS by 0.106 and improving Imaging Quality by 0.16. At four steps, consistency initialization mainly helps DMD converge faster: removing it increases LPIPS by 0.093 after 200 iterations, but the gap largely closes after roughly 1,200 additional iterations. This behavior is consistent with the observations in Causal Forcing++ regarding chunk-wise generation. Under the same backbone, data, teacher, and four-step DMD settings, a bidirectional student reaches similar per-frame quality by 1,200 iterations but yields higher LPIPS across budgets due to scene drift from the initial frame, and can output its first frame only after denoising the full 77-frame window.

Finally, the new system measurements provide a more complete picture of interaction performance. Including VAE decoding, ForgeWM-1 produces the first frame 135 ms after a chunk's controls are committed and sustains approximately 38 FPS. In our action-switching tests, yaw reversals are reflected at the commanded frame, including switches within a chunk, while the generated motion scales approximately linearly with the yaw magnitude (log–log slope 1.11).

Manuscript revision. We have revised the introduction to better reflect the scope of our contribution, clarified action alignment and cache updates in Section 3.2, and updated the discussion of consistency initialization in Section 3.3. A new Section 4.4 and Table 3 summarize the controlled training comparisons. Appendix F provides the full results, convergence curves (Fig. F1), motion analysis and failure cases (Figs. F2–F4), end-to-end latency and control measurements (Tables F5–F6), and the Matrix-Game 3.0 comparison (Table F7), and the controlled causal-vs.-bidirectional student comparison (Appendix F.9: Table F8, Figs. F5–F6). We have also clarified in Section 4.3 that replay refinement is intended for offline replay and export, rather than improving online interaction itself.

Dear Reviewers,

As the initial review scores for this submission are somewhat divergent, it is important that we thoroughly discuss our perspectives before reaching a final recommendation.

Now that the authors' rebuttal is available, I kindly ask each of you to respond in the discussion and clarify whether your original concerns have been addressed or still remain.

Thank you in advance for your timely engagement before the discussion period ends.

Best, Your AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.