- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
ConsisWorld evaluates spatial and control consistency in interactive video world models. It uses fixed reference geometry to measure scene preservation, and estimates each model’s action effects from its first four steps to evaluate sustained motion and action composition. The benchmark includes 140 initial views from 30 scenes and evaluates ten models. The results suggest that good visual or geometric scores do not necessarily imply stable camera control.
Critical strengths
-
Useful separation of evaluation goals. The paper evaluates geometry preservation, sustained motion, and action composition separately. This is more informative than judging generated videos mainly by their appearance. The results also show meaningful differences between models that would be difficult to identify from visual quality scores alone.
-
Careful measurement controls. Using separate anchors for pose fitting and geometry evaluation avoids directly evaluating the points used to fit the camera. The known-motion UE5 tests, support statistics, and sensitivity analyses are also useful. These checks make the evaluation more transparent than simply reporting final model rankings.
Critical weaknesses
-
Measurement validity on generated videos needs stronger support. The UE5 controls show that tracking and pose recovery can be accurate on rendered videos with known motion. However, they do not fully show how the scores respond when generated textures, visibility, or local shapes change while camera motion remains fixed. Tracking errors may then affect both geometry and action scores. Appendix H.5 also reports no whole-video motion-matched AlayaWorld/Lyra pairs, so their Ray comparison does not isolate geometry preservation under equally difficult motion. The support analysis is helpful, but does not fully resolve this concern.
-
The action reference assumes a narrow form of consistency. Extrapolating one fixed transform as (U_a^k) tests agreement with constant-step motion. A camera with repeatable acceleration or inertia could receive a low score even when its behavior is predictable. This matters because Table B.1 mixes keyboard controls, explicit camera increments, and text prompts. The native durations also differ: sixteen steps cover roughly two seconds for HY-World 2.0 but eleven seconds for H3-World. Motion normalization does not remove these differences. The conclusions should distinguish constant-step control from broader physical or temporal consistency.
Questions for authors
-
Measurement validity. Which existing controls separate tracking sensitivity from actual geometry changes in generated videos? Can the current comparisons be broken down by similar motion extent or duration?
-
Action interpretation. How does the benchmark distinguish unstable execution from repeatable acceleration? What evidence supports using a fixed action transform across the different control interfaces?