- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
SpaceTimeBench evaluates whether video models preserve spatial displacement and temporal progression when generating a broader-view version of an input video. It uses a sphere and a rotating pinwheel as measurement references, applies seven sequential checks, and reports results on 17 models across 350 prompts.
Critical strengths
- Cross-video event consistency is a useful evaluation target that complements conventional video-quality assessment.
- The staged checks provide useful diagnostics by separating event loss, reference-object failures, and spatial or temporal discrepancies.
Critical weaknesses
-
The evaluation relies on models generating a broader view of the same event from a text prompt, but it does not clearly verify whether the viewpoint actually changes as intended. Some models may retain a view close to the original, while others may generate substantially different views. This could affect the measured cross-view consistency and make comparisons across models less reliable.
-
The reliability of the spatial and temporal measurements is unclear. The projected rotation angle of the clock may vary with the viewing direction, even when the underlying rotation is unchanged. In addition, the variance of an object's 3D point cloud depends on its shape and size, not only on tracking uncertainty. These factors may affect the estimated spatial and temporal consistency and lead to incorrect failure diagnoses.
Questions for authors
-
How do you verify that the generated video shows the intended change in viewpoint? For example, how do you distinguish a genuine change in camera viewpoint from simply zooming out or making the original scene smaller in the frame?
-
Have you evaluated the pipeline on multi-view videos with known camera parameters, 3D geometry, and temporal information? If so, how often does it incorrectly identify physically consistent videos as failures?
-
In Table 4, are the baseline-positive samples selected according to WorldModelBench's judgments or the F1–F5 checks in SpaceTimeBench? Since the paper defines FDR based on positive results from WorldModelBench, please clarify how these samples are selected and how the denominator is calculated.
-
Table 4 extends beyond the right edge of the page. Please resize the table to fit within the page margins.