acceptodds
Withdrawn from ICLR 2027

ConsisWorld: Benchmarking Spatial and Control Consistency in Interactive Video World Models

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Interactive video world models allow users to explore generated environments through movement and turning commands. Reliable interaction requires both consistent scene structure across viewpoints and the ability to anticipate subsequent action effects from prior experience. We introduce ConsisWorld to jointly evaluate spatial and action consistency through three tests: scene geometry preservation, single-action execution, and multi-action composition. The benchmark contains 140 initial views from 30 synthetic and real-world scenes, covering 36,400 videos generated by ten models. The spatial test constructs a fixed 3D reference from the initial image, depth, and camera parameters, and checks whether generated views remain consistent with the original scene geometry as the camera moves. The action tests execute each of six basic actions repeatedly, estimate its single-step displacement and rotation from the first four steps, and use these estimates to predict subsequent motion and the endpoint relations of independently generated composite routes. Each model is thus evaluated against its own measured action effects. We find that some models preserve scene geometry well yet exhibit substantial motion deviations during repeated execution of the same action. Composing different actions exacerbates these consistency challenges, causing actual endpoint relations to deviate significantly from the predictions of isolated actions. Moreover, some models are stable across repeated generations from the same initial view, but the displacement induced by an action changes substantially across initial views. These results show that good scene preservation does not imply stable continuous control. There remains substantial room for improvement in reducing motion deviations during sustained execution and action transitions, and improving control consistency across initial views. ConsisWorld evaluates scene preservation and action execution together, providing a benchmark for comparing model capabilities and assessing future improvements.

Primary area:
datasets and benchmarks
Keywords:
Interactive World Models, Video World Models, World Model Evaluation

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_xQzJOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

ConsisWorld evaluates spatial and control consistency in interactive video world models. It uses fixed reference geometry to measure scene preservation, and estimates each model’s action effects from its first four steps to evaluate sustained motion and action composition. The benchmark includes 140 initial views from 30 scenes and evaluates ten models. The results suggest that good visual or geometric scores do not necessarily imply stable camera control.

Critical strengths
  1. Useful separation of evaluation goals. The paper evaluates geometry preservation, sustained motion, and action composition separately. This is more informative than judging generated videos mainly by their appearance. The results also show meaningful differences between models that would be difficult to identify from visual quality scores alone.

  2. Careful measurement controls. Using separate anchors for pose fitting and geometry evaluation avoids directly evaluating the points used to fit the camera. The known-motion UE5 tests, support statistics, and sensitivity analyses are also useful. These checks make the evaluation more transparent than simply reporting final model rankings.

Critical weaknesses
  1. Measurement validity on generated videos needs stronger support. The UE5 controls show that tracking and pose recovery can be accurate on rendered videos with known motion. However, they do not fully show how the scores respond when generated textures, visibility, or local shapes change while camera motion remains fixed. Tracking errors may then affect both geometry and action scores. Appendix H.5 also reports no whole-video motion-matched AlayaWorld/Lyra pairs, so their Ray comparison does not isolate geometry preservation under equally difficult motion. The support analysis is helpful, but does not fully resolve this concern.

  2. The action reference assumes a narrow form of consistency. Extrapolating one fixed transform as (U_a^k) tests agreement with constant-step motion. A camera with repeatable acceleration or inertia could receive a low score even when its behavior is predictable. This matters because Table B.1 mixes keyboard controls, explicit camera increments, and text prompts. The native durations also differ: sixteen steps cover roughly two seconds for HY-World 2.0 but eleven seconds for H3-World. Motion normalization does not remove these differences. The conclusions should distinguish constant-step control from broader physical or temporal consistency.

Questions for authors
  1. Measurement validity. Which existing controls separate tracking sensitivity from actual geometry changes in generated videos? Can the current comparisons be broken down by similar motion extent or duration?

  2. Action interpretation. How does the benchmark distinguish unstable execution from repeatable acceleration? What evidence supports using a fixed action transform across the different control interfaces?

Reviewer_Y2vUOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper evaluates whether video world models can preserve scene geometry and produce predictable motion when executing actions. To evaluate scene preservation and action consistency, the authors introduce ConsisWorld, which uses fixed 3D references for geometry and measured model action effects to predict later motion. This benchmark consists of 24 UE synthetic scenes and 6 ETH3D real-world scenes with 140 initial views. Experiments on ten models show that good geometric consistency does not guarantee stable control: action effects can vary over time, across action combinations, and between initial views.

Critical strengths

This paper studies an important problem in current video world models. Predictable action effects are essential for reliable interaction and planning. The paper evaluates whether these effects remain consistent under sustained and combined actions, addressing a basic requirement for usable video world models.

Critical weaknesses
  1. The novelty of this paper is overclaimed. This paper omits some important and related references (MIND[1], WorldRoamBench[2], and WorldExam [3]), which are similar to the current work by evaluating consistency, sustained action following, and motion calibration.

  2. It also lacks comparisons with existing consistency metrics on the same videos. It remains unclear which failures the new metrics detect that existing metrics miss.

  3. Control scores rely on camera motion estimated from generated videos. Scene distortions may make these estimates inaccurate, lowering scores even when control is consistent. The validation on rendered UE5 videos does not fully address this concern. The authors could keep camera motion fixed while adding controlled visual distortions, and separately test known changes in action effects. Comparing scores from estimated and ground-truth camera poses would help distinguish measurement errors from actual control failures.

  4. The composition tests use only one random seed. It is unclear how much of the prediction error comes from combining actions and how much comes from random variation. Repeating the tests with multiple seeds would help clarify this.

References:

[1] Yixuan Ye et al. MIND: Benchmarking Memory Consistency and Action Control in World Models. 2026.

[2] Ting-Bing Xu et al. WorldRoamBench: An Open-World Benchmark for Long-Horizon Stability of Interactive World Models. 2026.

[3] Yuxue Yang et al. WorldExam: Benchmarking World Models from Apparent Appearance to Inherent Reactivity. 2026.

Questions for authors
  1. Can you tell the more difference between your proposed metrics and other related papers I mentioned above?

  2. On the same videos, can you show failures that your metrics detect but existing consistency metrics miss?

  3. How do you distinguish actual control failures from errors in camera-motion estimation and calibration, especially when generated scenes deform?

Reviewer_SNS3OpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces ConsisWorld, a benchmark for evaluating spatial and action consistency in interactive video world models. It assesses three dimensions: scene geometry preservation, sustained single-action execution, and multi-action composition. The benchmark estimates each model's short-term action effects and uses them to predict subsequent motions and composite-route endpoints. Experiments on 140 initial views from 30 scenes, involving 36,400 videos generated by ten models, reveal that strong geometric consistency does not necessarily imply stable or predictable action control.

Critical strengths
  • Meaningful and comprehensive evaluation framework: ConsisWorld jointly evaluates spatial preservation, sustained action execution, and multi-action composition, capturing important aspects of interactive world modeling beyond visual quality and trajectory following. Its self-calibrated action reference also accommodates heterogeneous control interfaces.

  • Extensive experiments and methodological validation: The benchmark evaluates ten models across synthetic and real-world scenes using 36,400 videos. Additional pose-recovery validation, support diagnostics, and sensitivity analyses strengthen the credibility of the evaluation and reveal meaningful failure modes.

Critical weaknesses
  • Ambiguity in interpreting action-composition errors: The work predicts composite trajectories using fixed action transformations estimated from the first four steps. However, deviations may arise from predictable state-dependent dynamics or errors in estimating the initial transformations, rather than inconsistent action execution alone. The benchmark does not fully distinguish these factors, potentially limiting the interpretation of its composition scores.

  • Dependence of evaluation scores on motion and observation validity: Composition scores combine prediction accuracy with validity support, assigning zero to unsupported routes. Although the authors provide separate support statistics and sensitivity analyses, changing the motion threshold substantially affects several models' overall scores. Furthermore, Ray is evaluated on successfully tracked reference points, while different camera-motion magnitudes affect the evaluation difficulty. These factors complicate interpreting the scores as isolated measures of consistency.

Questions for authors
  • How does the benchmark distinguish genuinely inconsistent action execution from predictable state-dependent motion that cannot be captured by a fixed SE(2) transform? Can the authors provide evidence that the observed deviations primarily reflect undesirable inconsistency rather than limitations of the reference model?

  • Appendix H.3 shows that Ideal achieves higher translation scores than Full for most models, with differences concentrated in inverse-action routes. What additional failure modes or diagnostic insights does Full capture beyond the simpler Ideal reference, and how do these justify its use as the primary composition metric?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.