acceptodds
Withdrawn from ICLR 2027

FutureWorld: A Plug-and-Play Causal Next-State Predictor for Interactive Video World Exploration

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Interactive video world models aim to follow user actions while maintaining scene consistency over extended interactions. History-based methods reuse generated latents to preserve scene content, but three limitations of this reuse contribute to error recirculation. 1) The latent generated at each interaction is retained as history, potentially containing blurred details, geometric distortions, or semantic drift. 2) Although retrieved historical content matches the current scene, it can still contain errors that are carried into the next chunk. 3) New outputs then become history for later interactions, potentially reinforcing the same errors through repeated reuse. To limit this cycle, we aim to predict the future scene state from the current generated latent and the next action before generating the next interaction's video chunk. This future-state prior provides guidance for the next chunk, with the aim of reducing the errors carried forward from history. To this end, we propose a plug-and-play Causal Next-Chunk Predictor (C-NCP) with cross-model adaptability and prediction-guided modulation. We integrate C-NCP into four base generators and evaluate the augmented models across three benchmarks for interactive long-video generation and action-conditioned world modeling. Extensive experiments demonstrate reduced error accumulation and improved long-horizon consistency across diverse generation settings.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Video World Models, Next-State Predictor

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_SudKOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes a Causal Next-Chunk Predictor for improving consistency in interactive video generation. The predictor learns action-conditioned transitions in VAE latent space and guides frozen video generators through channel-wise residual modulation. A shared predictor trunk and VAE-specific adapters support integration with different generators. Experiments report improved quality, consistency, and interaction scores across several generators and datasets, supported by qualitative comparisons, component ablations, and a user study.

Critical strengths

S1. The modular approach has practical value for improving existing video world models. Maintaining scene content through continued interaction is a meaningful problem, and a reusable predictor offers a way to improve supported generators without retraining their generative backbones. The shared trunk and VAE-specific adapters make this contribution more useful than an improvement restricted to one generator.

S2. The empirical improvements extend beyond a single model or a visual-quality metric. The paper reports gains across generators and evaluation datasets, including interaction and consistency measures on WBench. The user study also considers action adherence and scene consistency through turns and revisits. Together, these results provide encouraging evidence that the plugin benefits the generated experience.

Critical weaknesses

W1. The methodological motivation should be further strengthened. The paper aims to reduce the propagation of errors in generated history by introducing a predicted future state as guidance. To this end, it uses the channel-statistic differences between the predicted next state and the current state to modulate the next state being generated. However, these differences describe expected scene changes rather than errors in the generator's output. The method does not distinguish changes already correctly produced by the generator from discrepancies that require correction. The connection between this modulation design and the motivation of reducing error propagation therefore remains insufficiently justified.

W2. The evaluation leaves the claimed reduction in error accumulation insufficiently established. The quantitative comparisons summarize rollouts of a fixed duration, while the qualitative examples show selected trajectories. WBench and the user study provide relevant evidence of improved consistency and interaction, but the reported results do not separately establish how scene fidelity deteriorates as interactions accumulate. A method could improve local appearance and continuity throughout a rollout while leaving the underlying rate of identity or layout drift largely unchanged. For sustained world exploration, whether a previously established scene remains reliable after repeated actions and revisits is more consequential than a higher aggregate score alone. Evidence of when such failures arise and how they develop would substantially strengthen the paper's central contribution.

Questions for authors

See Weaknesses.

Reviewer_kLzQOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes FutureWorld, a plug-and-play causal next-chunk predictor (C-NCP), to address the issue of error accumulation. It is trained using a two-stage schedule, predicting the next-chunk conditioned on current chunk and action.

Critical strengths
  1. The paper proposes C-NCP to address error accumulation in interactive video generation. It achieves consistent performance improvements with only modest inference overhead.
  2. The ablation studies demonstrate the effectiveness of the two-stage training strategy. The method is also evaluated on four different base models, including Matrix-Game 3.0, HY-World 1.5, LingBot-World-Fast, and SANA-WM, demonstrating its generalizability across different models.
Critical weaknesses
  1. The authors should compare their method with directly fine-tuning the base model using the same data . It is unclear whether the improvements come from the proposed module (C-NCP) or simply from additional training.
  2. The method predicts a complete future latent but only uses its channel-wise statistics for modulation, discarding spatial and temporal information. Why not use the full prediction? Is there a reason behind it, or can the network structure be better designed?
  3. There is still a Training–inference mismatch. Teacher forcing is used during training. The predictor is trained on ground-truth or its own predicted latents, but receives outputs from the base generator during inference.
  4. Most components are based on existing techniques. The overall contribution appears relatively incremental.
Questions for authors

Same as the Weaknesses.

Reviewer_4rjVOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes FutureWorld, a plug-and-play causal next-chunk predictor that forecasts future VAE latents from the current generated chunk and upcoming actions, and guides a frozen video generator through channel-wise modulation. Experiments across four base generators and three benchmarks report improvements in visual quality and temporal and scene consistency.

Critical strengths
  1. The paper addresses an important practical problem: maintaining consistency during sustained interactive video generation. The modular design supports multiple VAE latent spaces and integrates with frozen generators, making the approach potentially useful across different systems.

  2. The evaluation covers four base generators and three benchmarks, with consistently positive reported improvements. Component and training-schedule ablations, together with qualitative comparisons and a user study, provide supporting evidence for the approach’s empirical utility.

Critical weaknesses
  1. The proposed error-correction mechanism is insufficiently established. The predictor conditions on the current generated chunk, which may already contain accumulated errors; restricting its input to recent content does not inherently provide a more reliable reference than the generator’s historical context. For example, under progressive appearance drift, recent chunks may be more corrupted than some earlier retrieved views. Although Stage II includes predictor-generated feedback, this differs from the generator-produced inputs encountered at inference, and within-chunk teacher forcing introduces an additional mismatch. The paper appropriately acknowledges that its conditional-mean analysis does not guarantee rollout error contraction, but it lacks direct empirical evidence showing when the predictor corrects, preserves, or amplifies errors in generated inputs. Aggregate quality improvements alone do not establish the proposed correction mechanism.

  2. The statistical reliability and quality–latency trade-off require further evaluation. Although the method consistently improves the reported results across generators and benchmarks, the average-score gains are generally around 1.7–2.5 points on a 100-point scale. These improvements are not sufficiently large to dismiss potential variation due to training and sampling randomness without quantifying that variation. Experiments with additional independent training and inference seeds, together with means, standard deviations, and paired confidence intervals, are therefore needed to establish the robustness of the gains. Furthermore, Table 3 reports a 9.24% latency increase, or 0.617 seconds per interaction, for one base generator. Comparisons under matched inference budgets and overhead measurements across the other generators would help determine whether the improvements justify the additional computation and latency.

Questions for authors

According to Eqs. (12)–(13), the predicted future chunk influences generation only through its per-channel means and standard deviations aggregated over space and time. Is predicting the complete future latent necessary, or could a smaller action-conditioned network that directly predicts these statistics achieve comparable improvements? A comparison with such a baseline would help establish whether the gains justify the complexity of full future-state prediction.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.