- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper introduces WorldProbe, a benchmark that asks whether video world models remain coherent when exploration responds to what they generate. A goal-conditioned agent chooses camera actions online, assisted by target tracking and local geometry estimates. The benchmark contains 164 cases covering target approach, open-ended traversal, cross-view inspection, and successive region exploration, with task budgets up to 180 seconds. Experiments on ten models examine local quality, spatial memory, and evolution fidelity. The results document failures in scene persistence and coherent evolution during extended exploration, including cases where individual frames remain visually plausible.
Critical strengths
The exploration tasks provide a useful way to test whether a generated world supports sustained, goal-directed navigation. Adjusting actions to newly revealed geometry can help an evaluator continue toward a meaningful observation point when a predefined sequence would no longer reach it. The four task families give this idea a concrete and reusable evaluation setting.
The study across ten models provides useful examples of how local visual plausibility can coexist with failures in spatial memory and scene evolution. The separate analyses and accompanying visual examples help characterize these failures beyond a single aggregate quality score, even though the reliability of the automatic judgments needs further validation.
Critical weaknesses
The experiments do not establish the additional diagnostic benefit of adaptive exploration over predefined trajectories. There is no controlled comparison using the same scenes, models, generation budgets, and scoring procedure. The extended rollout in Section 4.4 demonstrates that the agent can keep operating, but does not establish its reliability across the benchmark or show that adaptation exposes more failures. The observed inconsistencies are useful evidence, but they could also arise under sufficiently long scripted exploration
Questions for authors
Could you compare adaptive exploration with predefined trajectories on a representative subset, keeping initial scenes, models, generation budgets, and scoring consistent? I would be interested in whether adaptation improves valid task completion or increases the number and variety of verified world-model failures.