acceptodds
Under review as a conference paper at ICLR 2027

WorldCast: Exploring Intelligent Visual Generation in World State Prediction

Abstract

Humans interpret multimodal observations of the world and forecast how it will evolve. Recent multimodal generative systems have demonstrated similar capabilities: they can integrate multimodal context and synthesize visual outputs that depict changes in the world. To characterize this emerging trajectory, we organize model capabilities into a four-level hierarchy and discover that higher-level tasks remain relatively underexplored despite model progress. To bridge this gap, we explore model performance on multi-step world state prediction, a high-level task that requires models to forecast successive future world states from multimodal observations and diverse references. Such a task requires models to possess entangled understanding and generation capabilities, as well as extensive world knowledge to translate human action sequences, physical transformations, and abstract visual reasoning into coherent visual outputs. We introduce WorldCast, comprising 320 manually designed cases across eight categories, and evaluate 12 representative models. The conducted experiments yield three key insights: (a) advances in frontier models are rapidly pushing the boundaries of intelligent world modeling, with the GPT-Image-2.5 System scoring 72.8 and demonstrating early-stage visual state prediction capabilities on real-world tasks; (b) WorldCast reveals substantial gaps between models: even Nano Banana Pro reaches only 42.8 and the best open-source model scores 20.1, with deficits appearing through complex context, entangled references, and reasoning-heavy tasks; (c) strong generative models effectively draw on the capabilities of their underlying language models. These findings highlight the need for multimodal generative models that are capable of linguistically structured planning and reasoning while also being deeply unified to faithfully simulate real-world states.

Primary area:
datasets and benchmarks
Keywords:
Benchmark, Image Generation, Visual Generation

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_otsSOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

WORLDCAST contains 320 cases across eight task categories to evaluate multistep visual state generation from heterogeneous references. The paper compares 12 systems and analyzes language enhancement, interleaved generation, and tool-assisted iteration. Human comparisons, multiple judges, and repeated generations validate the evaluation.

Critical strengths
  1. Meaningful tasks. The benchmark tests reference binding, action ordering, and object consistency across multiple steps, exposing failures that static image evaluation can miss.
  2. Multiple validation checks. Human comparisons, different judges, and repeated generations support the stability of the main performance gaps.
Critical weaknesses
  1. Unclear contribution. WorldReasonBench [1] already evaluates world-state prediction, while UEval [2] covers procedural multimodal generation. The paper does not clearly establish the additional diagnostic value of combining heterogeneous references with multistep generation. L3/L4 also lack testable classification criteria.
  2. Confounded experiments. Table 2 does not clearly control resolution per state or generation budget. The drop of 22.7 points therefore cannot be attributed directly to interleaved reasoning. Declining accuracy at later steps also leaves step difficulty and error propagation unresolved.
  3. Insufficient calibration against negative controls. A sequence can score 20 even with zero task completion and causal plausibility, provided visual consistency receives full credit. Without repeated-frame or shuffled-sequence baselines, the advantage over degenerate strategies and sensitivity to stalled or incorrectly ordered processes remain unclear.

Reference
[1] WorldReasonBench: Human-Aligned Stress Testing of Video Generators as Future World-State Predictors.
[2] UEval: A Benchmark for Unified Multimodal Generation.

Questions for authors
  1. Which failure modes does WORLDCAST reveal that existing benchmarks cannot readily detect? What criteria distinguish L3 from L4?
  2. When only the referenced physical property or rule changes, while the scene and instruction remain fixed, does the output change correctly? Are paired results available?
  3. Does the performance gap in Table 2 persist after controlling resolution per state and reasoning and generation budgets?
  4. After supplying correct historical states to remove error propagation, does accuracy still decline as context length increases?
  5. Where applicable, what scores do repeated initial frames or repeated target references receive on each metric and overall?
  6. For human-verified correct sequences, do task completion and causal plausibility scores consistently decrease when steps are shuffled or critical stages are removed?
9 comments
AuthorsOpenReview ↗

Response to Reviewer otsS (1/4): Contribution of WorldCast

We thank the reviewer for the detailed and constructive comments, and for recognizing the meaningfulness of our tasks and validation checks. Our responses are below, and we kindly ask the reviewer to consider raising the score if they find them helpful.

W1 & Q1. What is the contribution of WorldCast beyond existing benchmarks?

The core contribution of WorldCast is its focus on dynamically entangled multimodal understanding and generation. UEval [1] targets unified multimodal generation, with a focus on knowledge and verbal reasoning. WorldReasonBench [2] evaluates video generators that start from an initial world state (TI2V). WorldCast studies how visual understanding, textual reasoning/planning, and generation are dynamically coupled across multiple generation steps, which neither benchmark addresses. Heterogeneous references are provided precisely to probe this coupled capability.

This problem remains under-studied, as prior work mainly examines whether a model possesses certain knowledge, or whether it can faithfully reproduce pixel-level content (appearance, identity, style) from references. In WorldCast, references and task types are designed so that a reference can carry rules, annotations, target states, or physical properties that should take effect at different time steps. This tests whether a model can integrate several capabilities: understanding the reference, planning when and how to apply it, and generating accordingly, sometimes even with agentic behavior. WorldCast thus provides a more holistic evaluation of multimodal intelligence.

Under this focus, WorldCast exposes distinct diagnostic patterns, of which we highlight two representative ones. First, frontier models still often fail to utilize complex references. In Figure 4 (right), the target reference specifies the structure of the windmill, and the task requires attaching the blades in a specific order: even GPT-Image-2.5 drifts from the referenced shape at the first step, and Wan 3.0 attaches the blades in an incorrect order. Many models also fail to reflect complex physical properties specified by references, such as adhesion (Figure 14). Second, different combinations of components behave differently under the stress of heterogeneous references and multi-step generation. Such challenging settings require tightly coupled capabilities, and in particular strong language-side reasoning to orchestrate them; WorldCast diagnoses how these components interact with the generator through language-model enhancers (Section 4.3.1), interleaved generation (Section 4.3.2), and agentic harnesses (Section 4.3.3, Table 3).

References

[1] Bo Li et al. UEval: A Benchmark for Unified Multimodal Generation. arXiv:2601.22155, 2026.

[2] Keming Wu et al. WorldReasonBench: Human-Aligned Stress Testing of Video Generators as Future World-State Predictors. arXiv:2605.10434, 2026.

AuthorsOpenReview ↗

Response to Reviewer otsS (2/4): L3/L4 Criterion and Paired Reference Variants

W1 & Q1 (cont.). What distinguishes L3 from L4?

We agree that the L3/L4 criterion deserves a more explicit statement. As Figure 3 shows, L3 and L4 differ in task setting, specifically in the role that the references play. We formalize the difference as follows:

L3L4
Criterion: can the reference directly serve as the natural starting state?YesNo
Role of the referenceThe starting world state, to be observed and rolled outCarries information beyond the starting state (annotations, rules, target states, physical properties)
What the model must do firstObserve the initial stateReason over the information-bearing references

As noted in Section 2, L3/L4 classifies the complexity of task settings rather than ranking difficulty. We will state this criterion explicitly in the revision.

Q2. Do outputs change correctly when only the referenced property or rule changes?

We thank the reviewer for this valuable question, which we also took into account when designing WorldCast. If a model actually uses a reference, its output should change accordingly when the referenced property or rule changes; if it does not, the model has ignored the reference, and this should be penalized by our judge. In designing the cases, we intended that a natural rollout without relying on the references would generally not lead to the intended outcome, and the judge-only hints specify the states that the references determine. Outputs that neglect a reference therefore fail to match these states and lose credit.

In most cases the reference is an integral part of the task itself, so replacing it directly would make the task implausible or require redesigning the entire case. As the reviewer's question concerns physical properties or rules, we focused on cases in this domain and selected 5 of them in which the reference can be isolated relatively cleanly, constructing 10 paired variants in total. In 9 of the 10 pairs, the outputs change noticeably, and 8 of these changes are as expected; in the remaining two pairs, the output either does not reflect the change or reflects it incorrectly, which exposes a limitation of the model's own capability. We have updated the supplementary material with visual comparisons for all 10 pairs (ref_compare.pdf). If the reviewer finds this analysis informative, we will add it to the appendix.

AuthorsOpenReview ↗

Response to Reviewer otsS (3/4): Budget Control and Error Propagation

Q3 & W2 (part 1). Does the Table 2 gap persist after controlling resolution and budget?

We did control output resolution; the detailed settings are as follows, and we will document them in the appendix. For most four-state outputs, we use a 2× aspect ratio so that the output resolution per state can be matched. For certain input resolutions and for 2- or 6-state outputs, closed-source APIs only support specific resolutions, so we selected the closest available setting. As a benchmark, we control the final outputs and the way each model is invoked, whereas the internal reasoning and generation budgets of closed-source systems cannot be fully controlled.

Regarding the interpretation of the 22.7-point gap, our claim in Section 4.3.2 is that it reflects a difference in how the model is invoked, i.e., interleaved versus grid generation, rather than an effect attributable solely to interleaved reasoning. To address such concern directly, we further ran a controlled study on 198 cases in which the output budget is exactly matched between the two formats: the grid is generated at 2× resolution and then split into individual states, so each state has the same final resolution as the corresponding interleaved output. The gap narrows from 22.7 to 19.7 points but remains large:

GPT-Image-2.5InterleavedGridGain of interleaved
Overall74.354.6+19.7

Q4 & W2 (part 2). Does accuracy still decline when correct historical states are supplied?

We agree that separating step difficulty from error propagation could be of diagnostic value. Following the reviewer's suggestion, we ran this analysis on the 41 human-validated correct sequences (generated by GPT-Image-2/2.5, also requested from Q6), supplying Nano Banana Pro with those correct historical states. The step accuracy (%) is shown below:

Nano Banana ProStep 1Step 2Step 3Step 4
Original75.665.956.158.5
With correct history80.578.073.273.2

Accuracy improves but still declines across steps (80.5 → 73.2), suggesting that both error propagation and step difficulty contribute to the decline; the latter may arise from the increasing number of historical images at later steps, which adds to the information the model must process. However, we did not adopt this as the main protocol, for two reasons:

  1. Correct history cannot fully reproduce a model's own rollout. Open-ended tasks usually have no unique "correct" history. Even when correct historical images are supplied as suggested, the reasoning that produced them cannot be extracted, so this setting is not true teacher forcing. Moreover, for this controlled setting, each call requests the next state rather than all four states at once as in the original rollout, which may affect the model's internal generation pattern and partly explain the gap even at Step 1 (80.5 vs. 75.6).
  2. Our protocol is already aligned with what WorldCast aims to evaluate: coupled multimodal understanding and generation in realistic usage, where a model must build on its own previous outputs and reasoning. Evaluating the outcome of each model's own rollout directly reflects how understanding, reasoning, and generation jointly unfold across steps. This is also in line with related multi-step evaluation settings such as world-model benchmarks; for example, WBench [1] rolls out multi-turn interactions from each model's own previous outputs without supplying ground-truth intermediate states.

References

[1] Kaining Ying et al. WBench: A Comprehensive Multi-turn Benchmark for Interactive Video World Model Evaluation. arXiv:2605.25874, 2026.

Reviewer_hvdCOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a new benchmark to test whether a model can generate continuous, physically feasible scenes. It designs 320 cases across 8 task categories, requiring the generation of 1,352 future visual states, covering physical motion, robot manipulation, logical rules, and inferring intermediate steps from a target.

Critical strengths
  1. The research questions are valuable, addressing the shortcomings of existing evaluations and covering a wide range of topics.
  2. The evaluation metrics are comprehensive, assessing the generated results from three dimensions: task completion, causal plausibility, and visual consistency.
  3. The comparative experiments included 12 models, which is very comprehensive and demonstrates the applicability of the benchmark.
Critical weaknesses
  1. The biggest concern with this paper is the reasonableness of using GPT-5.5 for evaluation. In particular, GPT-5.5 itself lacks the ability to understand physical laws; how can it be used to evaluate the physical plausibility of the generated images?
  2. Image and video generation is often random; the same model may produce different results multiple times when faced with the same task. This paper lacks the necessary experiments to verify the challenges this randomness poses to different models.
  3. The final score of a paper is obtained by weighting TC, CP, and VC, with weights of 0.4, 0.4, and 0.2, respectively. However, these weights are manually set. For example, one model might perform well in task completion and causal plausibility but poorly in visual consistency; another model might do the opposite. Different weights can alter their final ranking.
  4. To unify the comparison between image and video models, this paper converts the video into ordered keyframes and then uses the same evaluation protocol. This approach facilitates standardized comparison, but it presents a significant problem: correct keyframes do not necessarily indicate correct motion between them. The advantages of video models in continuous motion modeling may not be fully realized.
Questions for authors
  1. How do the authors verify that the LMM judge can reliably identify subtle violations of physical laws and causal relationships?
  2. Will different prompts lead to different results from the LMM judge assessment?
  3. Will different weights affect the model's ranking? How should the weight combination be designed to best showcase the model's capabilities?
  4. How should we evaluate the quality of video generation models? I think videos generation model should have their own independent leaderboard, rather than being shared with image generation models.
4 comments
AuthorsOpenReview ↗

Title: Response to Reviewer hvdC (1/3): Reliability of GPT-5.5 as the Judge

We thank the reviewer for the constructive feedback and for recognizing the value of our research questions, metrics, and model coverage. We hope the following clarifications and new results address the concerns, and we kindly ask the reviewer to consider raising the score if they find them helpful.

W1 & Q1 & Q2. Can GPT-5.5 reliably judge physical laws and causal changes?

GPT-5.5 is already a strong judge for this task under our rubrics and task-specific hints. Our human-written hints (Section 3; see Appendix G for examples) are designed to turn judgments about physical laws and causal changes into grounding checks. For cases that involve physical dynamics, the hints state the observable consequence of the relevant physics. For example, in the Physical Simulation case in Appendix G (cutting the string of a submerged ball), the hint specifies how each object should move after the cut, so the judge mainly needs to check whether the generated steps show that movement. This turns a large part of "understanding physical laws and causal changes" into perception and grounding, which modern LMMs such as GPT-5.5 handle well.

A well-designed rubric reduces the burden on the judge, and this follows established practice: HarnessEval-W [1] also uses GPT-5.5 as its evaluator backend with case-specific evaluation sub-questions, and PhyGenEval [2] evaluates physical commonsense with an older judge (GPT-4o) under prompt-specific standards. We compared different judge models in Appendix E (Figure 12). The results are stable across judges, indicating that our rubrics and hints are good enough for different judges to yield highly consistent model-level rankings. This also answers Q2: since rubrics and hints strongly affect the judge, we evaluate all models with every LMM judge under the same rubrics and hints. The full prompt templates are provided in Appendix H. To further test whether the results depend on the specific wording, we used GPT-6 Astra to rephrase the rubric while preserving their semantics, and re-scored the outputs in Section 4.4 with GPT-5.5. The model ranking remains unchanged and pairwise winners under the original and rephrased rubrics largely agree (exact-match rate of 87.1%/89.2%/82.5%/78.5% for Overall/TC/CP/VC), indicating that our evaluation does not hinge on the particular wording of the rubrics.

Most importantly, GPT-5.5's judgments under our rubrics and hints are in line with human experts (Section 4.4, Figure 9). The model rankings given by GPT-5.5 correlate strongly with those given by human experts (r ≥ 0.97 for the overall score and every dimension), and GPT-5.5 agrees with the human-majority winner in ≥ 86% of valid pairwise comparisons. We further compare GPT-5.5 with another commonly used judge, Gemini 3.1 Pro, on the same valid human-majority winner votes, reporting the exact-match (EM) rate with the human majority:

JudgeOverall-EMTC-EMCP-EMVC-EM
GPT-5.5 (ours)86.5%87.4%89.2%86.1%
Gemini 3.1 Pro64.3%56.4%41.4%31.5%

While different judges agree on model-level rankings, GPT-5.5 is more reliable at the case level under our setting: it distinguishes between outputs more finely and is less prone than Gemini 3.1 Pro to assigning the same score to outputs of clearly different quality, which is why we adopt it as the default judge.

References

[1] Weiliang Chen et al. HarnessEval-W: Agentifying the Evaluation of Visual Worlds. arXiv:2608.16859, 2026.

[2] Fanqing Meng et al. Towards World Simulator: Crafting Physical Commonsense-Based Benchmark for Video Generation. arXiv:2410.05363, 2024.

AuthorsOpenReview ↗

Response to Reviewer hvdC (2/3): Generation Randomness and Weight Robustness

W2. Randomness of generation

We agree that this is important, and we have already evaluated it: Appendix E (Table 4) repeats generation for three rounds for all 12 models and scores every run with GPT-5.5. The standard deviation of the overall score never exceeds 0.53 (e.g., GPT-Image-2.5: 72.8 ± 0.39; Nano Banana Pro: 42.8 ± 0.48). Individual categories vary more, up to 2.7 points. These deviations are small relative to the gaps between most models. We apologize that this result appeared only in Appendix E, and we will move a summary of it into the main text.

W3 & Q3. Do different weights change the ranking, and how should the weights be designed?

Weights can change the relative order of a few adjacent models, but the overall ranking is generally stable. First, with GPT-5.5 as the judge on the full benchmark, the ranking of all 12 models is identical under our weighting (0.4/0.4/0.2), equal weights (0.33/0.33/0.33), and TC- or CP-heavy weightings (0.5/0.3/0.2 and 0.3/0.5/0.2):

GPT-Image-2.5 > GPT-Image-2 > Seedance 2.0 > Nano Banana Pro > Gemini Omni > Wan 3.0 > Seedream 5.0 Pro > SenseNova-U1 > Seedream 4.5 > Emu3.5 > BAGEL > Qwen-Image

Under 0.5/0.5/0, only two adjacent pairs swap places: Nano Banana Pro and Gemini Omni (4th/5th), and BAGEL and Qwen-Image (11th/12th).

Second, regarding how the weights should be designed to best reflect model capabilities, we use agreement with human experts as the criterion. Although model-level rankings are relatively robust across reasonable weightings, case-level agreement with humans differs. On the 100 human-annotated cases from Section 4.4 (Figure 9), 0.4/0.4/0.2 gives the highest agreement with human overall ratings, which supports our choice of it as the default. We will add this analysis to the main text:

Weighting scheme TC/CP/VCPearson r (w/ human)Spearman ρ (w/ human)
TC 0.40 / CP 0.40 / VC 0.20 (ours)0.8450.838
TC 0.33 / CP 0.33 / VC 0.330.8180.813
TC 0.50 / CP 0.30 / VC 0.200.8390.834
TC 0.30 / CP 0.50 / VC 0.200.8310.823
TC 0.50 / CP 0.50 / VC 0.000.7910.791
AuthorsOpenReview ↗

Response to Reviewer hvdC (3/3): Evaluating Video Generation Models

W4 & Q4. How should video generation models be evaluated?

WorldCast is designed to evaluate key-state prediction: whether a model produces the semantically critical world states of a task in the correct causal order. Motion continuity between key states is therefore not the primary target of this benchmark and we view WorldCast as complementary to video-centric benchmarks; as noted in Section 4.2, visual continuity does not necessarily translate into accurate key states. Nevertheless, we agree that a separate reference for video models is useful. To examine how continuity affects the results, we additionally evaluate video models on a fixed number of uniformly sampled frames (32) instead of only adapter-selected keyframes, and report the results as a separate video leaderboard:

OverallSeedance 2.0Gemini OmniWan 3.0
Keyframes (original)47.241.639.9
Uniform frames (separate leaderboard)45.243.541.1

Under the 32-frame protocol, the ranking among video models is unchanged and scores change by at most 2 points, indicating that our key-state evaluation does not systematically disadvantage video models. We will include this video leaderboard in the revision.

AuthorsOpenReview ↗

Kind Reminder

Dear Reviewer hvdC,

Thank you again for your time and valuable comments. As the discussion period ends on November 18, we would like to kindly check whether our responses have addressed your concerns. In particular, regarding your main concern, we provided further evidence supporting the use of GPT-5.5 as the judge, including its agreement with human experts and the robustness of our curated rubrics. We also addressed the concern about randomness, explained the rationale for the weight design, showed that the model ranking is generally stable across weightings, and clarified our video evaluation with supplementary results on uniformly sampled frames.

If you have any remaining questions, we would be happy to address them before the discussion ends. If our responses have resolved your concerns, we would sincerely appreciate it if you could consider updating your score.

Best regards,

The Authors

Reviewer_6XTPOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper introduces WorldCast, a benchmark comprising 320 cases and 1,352 visual states across eight task categories, designed to evaluate whether multimodal generative systems can produce coherent sequences of world states conditioned on heterogeneous visual and textual references.

Critical strengths
  • The paper identifies a meaningful gap in existing generative model evaluation. Most benchmarks emphasize visual quality, static semantic alignment, or temporal consistency, while WorldCast examines whether models can integrate heterogeneous references to produce logically and causally coherent sequences of visual states.
  • The benchmark covers eight categories that test substantially different combinations of physical knowledge, planning, spatial reasoning, and reference understanding, providing a multidimensional view of model capabilities rather than relying on a single type of generation task. Also, beyond benchmark rankings, the paper investigates factors influencing performance and provides valuable empirical analyses of language enhancement, interleaved generation, and agentic refinement.
Critical weaknesses
  • The distinction between L3 and L4 is based on the degree of integration among reference understanding, reasoning, and generation, without a sufficiently rigorous definition. There is no controlled empirical study demonstrating that L4 tasks systematically require more advanced intelligence than L3 tasks. Indeed, some frontier models achieve higher scores on individual L4 categories than on L3 categories.
  • Most tasks involve only four steps, and the paper does not evaluate performance over longer horizons. Such short sequences are insufficient to characterize important challenges in long-horizon world modeling, including accumulated prediction errors, long-range causal dependencies, and sustained physical dynamics.
Questions for authors
  • The evaluation score uses fixed weights of 0.4, 0.4, and 0.2 for TC, CP, and VC, respectively, but these values are not justified.
5 comments
AuthorsOpenReview ↗

Response to Reviewer 6XTP (1/3): Justification of the TC/CP/VC Weights

We thank the reviewer for recognizing that WorldCast addresses a meaningful gap in generative model evaluation and for the valuable suggestions. Our point-by-point responses are below.

Q1. Justification of the weights for TC/CP/VC

When designing the metric, we considered TC and CP to be more critical than VC in many tasks, and therefore assigned them higher weights. We thank the reviewer for raising this point and will explain the rationale more clearly in the revision. Prior works such as WorldReasonBench [1] and GENIUS [2] also assign fixed, unequal weights to their evaluation dimensions. To further validate our choice, we measured how well several common weighting schemes agree with human preferences on the 100 human-annotated cases from Section 4.4 (Figure 9):

Weighting scheme TC/CP/VCPearson r (w/ human)Spearman ρ (w/ human)
TC 0.40 / CP 0.40 / VC 0.20 (ours)0.8450.838
TC 0.33 / CP 0.33 / VC 0.330.8180.813
TC 0.50 / CP 0.30 / VC 0.200.8390.834
TC 0.30 / CP 0.50 / VC 0.200.8310.823
TC 0.45 / CP 0.45 / VC 0.100.8210.814
TC 0.50 / CP 0.50 / VC 0.000.7910.791

The results are consistent with our design rationale, and our weighting achieves the highest agreement with humans. TC and CP measure whether the predicted world states are correct, so they carry most of the weight. Shifting weight between TC and CP changes agreement only slightly (rows 3–4), so weighting the two equally is reasonable, whereas VC is necessary: down-weighting it (row 5) or dropping it entirely (last row) clearly reduces agreement with humans.

References

[1] Keming Wu et al. WorldReasonBench: Human-Aligned Stress Testing of Video Generators as Future World-State Predictors. arXiv:2605.10434, 2026.

[2] Ruichuan An et al. GENIUS: Generative Fluid Intelligence Evaluation Suite. arXiv:2602.11144, 2026.

AuthorsOpenReview ↗

Response to Reviewer 6XTP (2/3): Definition of L3 vs. L4

W1. Definition of L3 vs. L4

We agree that the L3/L4 criterion deserves a more explicit statement. As Figure 3 shows, L3 and L4 differ in task setting, specifically in the role that the references play. We formalize the difference as follows:

L3L4
Criterion: can the reference directly serve as the natural starting state?YesNo
Role of the referenceThe starting world state, to be observed and rolled outCarries information beyond the starting state (annotations, rules, target states, physical properties)
What the model must do firstObserve the initial stateReason over the information-bearing references

L3/L4 is not a difficulty ordering. As noted in Section 2, it classifies the complexity of task settings rather than ranking difficulty, so strong models may even perform better on L4 than on L3 tasks, which is consistent with the reviewer's observation. We will make this criterion explicit in the revision.

AuthorsOpenReview ↗

Response to Reviewer 6XTP (3/3): Horizon of Key-State Prediction

W2. Horizon concern

We agree that long-horizon world modeling is important. WorldCast focuses on multi-step key-state prediction, and many tasks require a model to understand multimodal observations and dynamically use the information to generate the critical states through entangled reasoning. In such settings, the more key states a task has, the more complex the temporal dependencies among them become, and the harder it is to judge the sequence reliably.

Our evaluation covers 2 to 6 steps, and most tasks use 4 steps. This horizon is already challenging and exposes clear performance gaps between models. It also reveals severe error accumulation even for strong models such as Nano Banana Pro, as shown in Figure 7 (Section 4.3). We therefore believe that the current horizon is a reasonable setting: it covers the capability range of current models while still yielding reliable scores. We will consider extending WorldCast to longer horizons as models improve.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Global Response

We sincerely thank all reviewers for their time and effort in reviewing our paper, and we are grateful for the encouraging comments. Reviewer 6XTP notes that our work identifies a meaningful gap in existing generative model evaluation and provides valuable empirical analyses. Reviewer hvdC finds the research questions valuable and the evaluation metrics comprehensive. Reviewer otsS describes our tasks as meaningful and capable of exposing failures that static image evaluation can miss. Reviewers 6XTP and otsS also find the paper clear and easy to follow.

We believe that multi-step world-state prediction is a natural testbed for examining how multimodal generative systems couple understanding, reasoning, and generation. Through WorldCast, we find that even frontier systems still often fail to utilize complex references to predict how world states evolve, and substantial gaps remain across models. Our analyses further suggest that performance depends not only on each individual capability, but also on how they are combined.

We have carefully addressed all of the reviewers' concerns in the corresponding responses, including:

  • the reliability of using GPT-5.5 as the judge, including the role of curated hints, as well as the judge's agreement with human experts and the robustness of our rubrics (Reviewer hvdC W1/Q1/Q2);
  • the justification of the TC/CP/VC weights and the stability of the ranking under alternative weightings (Reviewer 6XTP Q1, hvdC W3/Q3);
  • a testable L3/L4 criterion (Reviewer 6XTP W1, otsS W1/Q1) and the rationale for the current horizon (Reviewer 6XTP W2);
  • the contribution beyond WorldReasonBench and UEval, and paired reference variants (Reviewer otsS W1/Q1/Q2);
  • the randomness of generation (Reviewer hvdC W2) and a separate video leaderboard (Reviewer hvdC W4/Q4);
  • a resolution-matched interleaved vs. grid comparison, a correct-history diagnostic probe (Reviewer otsS W2/Q3/Q4), and negative controls with degenerate and perturbed sequences (Reviewer otsS W3/Q5/Q6).

Visual comparisons for the paired variants have been added to the supplementary material, and the other results will be incorporated in the final version.

We are grateful for the constructive follow-up discussion. Reviewer 6XTP has confirmed that our responses resolved their concerns, and Reviewer otsS raised their score to weak accept after further discussion. We have not yet heard back from Reviewer hvdC, and we would respectfully ask whether our responses have addressed the raised concerns. We would be happy to provide any further clarification.

Once again, we thank all reviewers and the AC for their time and thoughtful engagement with our work.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.