- Rating
- 2
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
In this paper, the authors propose CRONOS, an intervention-based benchmark with evaluation metrics for video generation. This benchmark considers three basic physical events and applies four controlled counterfactual visual interventions to them, and also presents several findings.
Critical strengths
-
The authors build a well-controlled benchmark with systematic interventions on camera viewpoints, scenes, object categories, and appearances for video generation models;
-
The authors evaluate various video generation models under I2V and V2V and investigate the impact of model size on physical consistency;
-
The paper provides human user study and finding, like shows visual changes can affect the physical consistency of generated videos.
Critical weaknesses
-
The writing is a bit verbose, like the long background in the Introduction and please add clear contribution summarization in introduction. The related works could be more focus on video generation benchmark and evaluation metrics.
-
This paper lacks comparisons with highly relevant works like PhyWorldBench[1] and Impossible Videos[2]. I not see any novel innovation because the concept "counterfactual physical consistency" very similar to the "Anti-Physics". This already presented in previous paper[1]. It is better to claim your novelty.
-
Only evaluating three simple physical events. While the authors argue these are fundamental interactions, a comprehensive standard benchmark needs much broader coverage.
-
The evaluation metrics (like motion similarity and appearance stability) simply adopt existing designs like VBench. The authors need to claim your metric design is uniquely suited for evaluating counterfactual physical consistency. And I have doubts about whether using DINOv2 [CLS] token can measure true appearance stability.
-
The experiments seem just translates the numbers from the tables into text. It lacks deeper insights or specific observations based on generated video results from different models. This makes the takeaways basic. And I recommend the author to expend your benchmark to include proprietary models.
[1] PhyWorldBench: A Comprehensive Evaluation of Physical Realism in Text-to-Video Models. ICLR2026.
[2] Impossible videos. ICML 2025.
[3] VBench. CVPR2024
Questions for authors
see weakness.
We thank the reviewers for their insightful feedback. After carefully considering their concerns, we have substantially revised the design of our evaluation metrics, resulting in updated results and findings. We believe these changes go beyond the scope of a rebuttal and have therefore decided to withdraw our submission. We thank the reviewers again for their time and valuable contributions to improving our work.