- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper presents CausalCascade, a causal cascade generation benchmark to evaluate whether I2V models can generate physically plausible events with causal dependencies sequentially. The benchmark includes 50 Blender scenes, structured event graphs, three-level future event prompts, and 2832 counterfactual variants. The authors evaluated nine I2V models on CausalCascade. The results show that more detailed prompts can improve event recovery, but the physical process, temporal coherence, and intervention response remain unreliable.
Critical strengths
- Future-event specification at three levels and Temporal alignment separately examine the impact of insufficient event information and temporal misalignment, which is more helpful in explaining the causes of failure than simply reporting the quality of the generated results.
- The authors consider four types of physical interventions and independently re-simulate each modified scene. The evaluation subset is selected without reference to model performance, and factual-counterfactual pairs use matched prompts. These choices make the pairwise comparisons more reliable.
Critical weaknesses
- CF Spatial Effect IoU and CF Preserve Spatial IoU seem to be unable to effectively differentiate between the model's fundamental generation errors and true causal response errors. Suppose a model already struggles to generate accurate factual rollouts; even if the model can respond to interventions, it may still fail to obtain the correct motion mask due to insufficient base generation quality.
- The benchmark seems to conflate fidelity to a specific simulator-generated rollout with causal correctness. It compares each generated video against a single reference rollout, potentially assigning low scores to alternative physically plausible futures that preserve the correct causal dependencies but differ in motion details or timing.
Questions for authors
Please see the weakness part.
6 comments
A Gentle Follow-up on Our Rebuttal
Dear Reviewer rCS3,
Thank you again for your time and valuable feedback. We are writing to ask whether our rebuttal has addressed your concerns, and we would be happy to discuss any remaining issues if needed.
A Second Follow-up on Our Rebuttal
Dear Reviewer rCS3,
We would like to briefly follow up on our previous message. We have put considerable effort into addressing the concerns raised in your review, including conducting additional experiments and incorporating the corresponding analyses and clarifications into the revised manuscript.
If you have had a chance to review our responses, we would greatly appreciate any feedback on whether the additional evidence has addressed your concerns, or if there are still any points that require further clarification. We would be glad to address them promptly.
We sincerely appreciate your time and consideration.
Best regards,
The Authors
Response to Reviewer rCS3
We sincerely thank the reviewer for recognizing the diagnostic value of our three-level future-event specification and temporal-alignment analyses, as well as for the thoughtful assessment of our factual-counterfactual evaluation design.
We thank the reviewer for this insightful observation regarding the interpretation of our motion metrics. It motivated us to further disentangle base-motion realization from conditional intervention response.
The original CF Spatial Effect IoU and CF Preserve Spatial IoU provide useful end-to-end measures of agreement with the expected intervention-related motion. We therefore retain these metrics while introducing a unified two-stage decomposition of the recall-side coverage of both Preserve and Effect into Base Realization (C) and Conditional Response (R).
For Preserve, let P∗ denote the ground-truth shared-motion map as defined in Appendix F. Base Realization (CP=∣P∗∣∣P∗∩M^f∣) measures how much of the ground-truth shared motion is realized in the factual generation; Conditional Response (RP=∣P∗∩M^f∣∣P∗∩M^f∩M^cf∣) measures how much of this realized motion is retained in the counterfactual generation.
For Effect, following Appendix F, let S∗ and A∗ denote the ground-truth motion maps to be suppressed and induced, respectively, and S^ and A^ their corresponding model generated motion maps.
In Table R3-1, all entries report Spatial metrics, following the original scene-macro aggregation and equal weighting across the four edit types and nine models. Complete definitions and corresponding weighted results have been provided in the Appendix H.3 of the revised PDF.
Table R3-1. Decomposition of Base Motion Realization and Conditional Intervention Response.
Notably, the results reveal an asymmetry between preserving existing dynamics and selectively modifying them. Together, these results highlight two complementary bottlenecks: incomplete base-motion realization and limited selective intervention responsiveness, with the latter persisting even after conditioning on realized target motion.
We thank the reviewer for raising this concern. CausalCascade evaluates physical process realization conditioned on the initial scene states and relevant physical properties provided in the L1/L2 prompts, rather than open-ended future prediction. Under these specified conditions, Blender provides a reproducible reference rollout with fixed simulation settings.
Our motion evaluation also allows for variations in motion timing. Both Spatial IoU (which aggregates motion masks over time) and Weighted Spatial IoU (which accounts for each pixel's temporal activation frequency) relax the requirement for frame-wise temporal correspondence. For Strict ST-IoU, which is sensitive to temporal alignment, our original evaluation (Finding 3) further compares strict temporal matching with global affine alignment and constrained DTW, accounting for global and local timing variations.
Complementarily, graph-grounded VQA evaluates event occurrence, causal pathways, temporal ordering, and outcomes without requiring the generated video to reproduce the full motion details and timing of the reference rollout.
We thank the reviewer again for the time and thoughtful consideration. If our responses have adequately addressed your concerns, we would sincerely appreciate your reconsideration of the score.
Sincerely,
The Authors