acceptodds
Under review as a conference paper at ICLR 2027

YoCausal: How Far is Video Generation from World Model? A Causality Perspective

10 views

Abstract

As video diffusion models (VDMs) advance toward world models, a key question arises: do they truly understand causality? Existing benchmarks mostly rely on synthetic data, limiting real-world generalization due to the sim-to-real gap. We present YoCausal, a two-level benchmark inspired by the Violation of Expectation (VoE) paradigm from cognitive science. By temporally reversing real-world videos at zero cost as natural counterfactual samples, YoCausal establishes an arbitrarily extensible evaluation protocol. Level 1 introduces the Reverse Surprise Index (RSI), quantifying arrow-of-time perception via denoising loss. Level 2 introduces the Causality Cognition Index (CCI), which leverages a VLM to stratify datasets into causal and non-causal subsets, disentangling genuine causal reasoning from temporal bias. Evaluation of 13 state-of-the-art VDMs reveals that perceiving the arrow of time does not imply understanding causality, and a significant gap persists relative to human-level causal cognition.

Primary area:
datasets and benchmarks
Keywords:
Video Generation, World Models, Causality, Violation of Expectation, Diffusion Models, Cognitive Science

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_uJxQOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces YoCausal, a benchmark for probing temporal and causal sensitivity in video diffusion models using forward and temporally reversed real-world videos. It measures forward-video preference through the Reverse Surprise Index (RSI), based on paired denoising losses, and introduces the Causality Cognition Index (CCI), which contrasts RSI between videos with and without salient causal interactions. Experiments on 13 open-source models reveal different rankings under the two metrics and gaps relative to human performance. The paper further examines agreement with human judgments of generated videos, sensitivity to the VLM used for dataset partitioning, and the effect of text conditioning.

Critical strengths
  1. The use of paired forward and reversed real-world videos provides a practical, extensible evaluation protocol while controlling for visual content within each pair. The two-level design addresses the important distinction between sensitivity to temporal direction and sensitivity to salient causal interactions.

  2. The paper supports its proposed metrics with complementary validation, including human evaluations of generated videos, alternative VLM partitioners, and a null-prompt ablation. The correlation with human judgments provides useful evidence that the benchmark captures behavior relevant to the causal plausibility of generated videos.

Critical weaknesses
  1. CCI’s interpretation as causal sensitivity is not sufficiently established, since similar optical-flow magnitudes do not rule out differences in event content or reversal difficulty between the two subsets. CCI can also approach zero when both subsets are recognized reliably, making low scores ambiguous despite the proposed rank aggregation.

  2. The stability of the reported scores and rankings under denoising-loss estimation remains unclear. Evaluation uses only ten sampled timesteps and one noise sample per timestep, without reporting sensitivity to random seeds or larger sampling budgets.

  3. YoCausal requires access to internal denoising predictions and therefore cannot directly evaluate closed-source models. This restriction reduces its generality and limits the applicability of its conclusions beyond the evaluated open-source models.

Questions for authors
  1. Do the CCI differences and model rankings persist when causal and non-causal videos are matched for event content, motion characteristics, and reversal difficulty? How should low CCI be interpreted when RSI is high on both subsets, and does combining CCI with RSI improve agreement with human judgments over RSI alone?

  2. How stable are RSI, CCI, and the aggregate rankings across random seeds and larger timestep/noise sampling budgets? Does the reported bootstrap account for variability from timestep and noise sampling, or only from resampling videos?

  3. Could the authors evaluate MiniMax-H3 to examine whether the RSI–CCI differences and human–model gap persist?

3 comments
AuthorsOpenReview ↗

Response to Reviewer uJxQ

We thank the reviewer for recognizing that paired forward/reversed clips control visual content, that the two-level design separates temporal direction from causal interactions, and that our validation (human evaluation, alternative VLMs, null-prompt ablation) shows the benchmark captures causal plausibility.

W1 / Q1 — Interpreting CCI.
Reversal difficulty. We agree the subsets differ in event content and causal videos can be easier to recognize as reversed. That extra detectability is the causal signal itself, not a confound: reversing a causal video creates a causal anomaly (e.g., shards reassembling) that only a model expecting cause before effect can exploit. Non-causal cues are matched or nearly matched:

  • Motion: nearly identical optical-flow magnitude distributions (Cohen's dd = 0.057 < 0.2; Fig. 8a).
  • Duration and captions: within each subset, clip duration is fixed and prompts share one captioning source and structure; as CCI is computed within subsets, both are matched by construction.
  • Scenes and objects: we used a VLM to label each video with a Places365 scene category and a COCO object supercategory; the subsets differ only slightly within every domain (Cramér's V≤0.28V \le 0.28, a small effect).

Model behaviour agrees: if the causal subset were simply easier, high-RSI models would also score high on CCI, yet RSI and CCI rank the 13 models with only τ = 0.13.

Saturation. We agree CCI could approach zero if both subsets were recognized reliably, but current models are still far from saturation, and CCI is designed to be read jointly with RSI rather than as an absolute score (Sec. 3.4).

Agreement with humans. In rank correlation, the aggregate ranking matches RSI alone (Kendall's τ = 0.667 for both), but this only reflects pairwise ordering. To test whether CCI adds information, we regress each model's human preference score (Table 7) on its metrics:

PredictorsR2R^2adj. R2R^2pp (gain over RSI)
RSI0.560.52–
RSI + CCI0.790.750.007

Adding CCI raises R2R^2 (variance in the 13 models' human preference scores explained) from 0.56 to 0.79 and adjusted R2R^2 from 0.52 to 0.75; a nested F-test confirms the gain is significant (pp = 0.007): CCI carries information about human-perceived causality that RSI misses.

W2 / Q2 — Seed and timestep stability.
RSI averages each pair's denoising loss over KK = 10 timesteps, with one Gaussian noise sample per timestep shared by both clips; CCI is causal-subset RSI minus non-causal-subset RSI. Directly: the bootstrap resamples videos only and excludes timestep and noise variability. We ran two new experiments on four models covering all three prediction targets: AnimateDiff-SD1.5 (ϵ\epsilon), CogVideoX-2B (vv), Wan2.1-1.3B and LTXV-2B (flow), otherwise following the paper.

Model (%)RSI, 5 seedsRSI, KK=5/10/15CCI, 5 seedsCCI, KK=5/10/15
AnimateDiff-SD1.544.91 ± 0.5444.37/45.69/45.75−5.29 ± 0.33−4.64/−5.18/−4.58
CogVideoX-2B46.70 ± 0.2147.13/46.89/46.65−0.25 ± 1.25−1.24/0.31/1.28
Wan2.1-1.3B45.93 ± 0.3046.42/45.51/47.195.10 ± 0.652.83/5.39/6.77
LTXV-2B56.75 ± 1.2356.43/58.86/56.44−0.71 ± 1.52−5.08/−0.16/0.72

The seed std is 0.30–1.23 points for RSI and 0.33–1.52 for CCI, i.e., moderate. From KK = 10 to 15, scores change by at most 1.4 points except LTXV-2B's RSI, so KK = 10 is sufficient. As Monte-Carlo estimates of the expected denoising loss, RSI and CCI would become more robust with more seeds given sufficient compute. Each extra seed or budget needs a full benchmark pass, so the larger models (5B–14B) would not finish within the rebuttal period on our 24 GB RTX 4090; we will add them in the revision once larger GPUs are available.

W3 — Closed-source models.
Loss access limits which models can be tested today, not the method: YoCausal needs only the denoising loss at a given timestep and noise seed, not the weights. Providers could expose a minimal loss-query interface without revealing internals, or run YoCausal internally to select checkpoints. Open-weight models are also quickly closing the gap to the state of the art, widening YoCausal's reach.

Q3 — MiniMax-H3.
We agree a stronger model would further test our findings. YoCausal evaluates every model in its official setting; H3's full-precision weights (~66 GB DiT plus a 51.5 GB text encoder) exceed our 24 GB RTX 4090. Even with sufficient memory, a 33B model would not finish the full benchmark within the rebuttal period. We will include H3 in the revision once larger GPUs are available.

We thank the reviewer again. If these responses address your concerns, we would be grateful if you would consider adjusting your score.

Sincerely,
The Authors

I thank the authors for their careful rebuttal, which has addressed most of my concerns. I therefore maintain my positive rating.

We sincerely thank reviewer uJxQ for the time and thoughtful feedback, and are glad our response addressed the concerns.

Reviewer_fFYNOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes YoCausal, a benchmark for evaluating temporal perception and causal understanding in video diffusion models. It compares denoising losses on forward and reversed real-world videos to compute the Reverse Surprise Index (RSI). It further introduces the Causality Cognition Index (CCI), defined as the RSI difference between causal and non-causal subsets identified by a vision-language model. The results show differing model rankings across RSI and CCI and a gap relative to human performance.

Critical strengths

The benchmark uses real-world videos and temporal reversal to construct evaluation pairs without requiring video generation models to generate new videos. This makes it straightforward to extend to additional domains and provides a practical way to evaluate video models across diverse scenes.

Critical weaknesses

The methodological novelty is limited. Using denoising losses to compare physically plausible and implausible videos was already explored by LikePhys (Yuan et al., 2025). Distinguishing forward from time-reversed videos is an established task for studying the arrow of time (Pickup et al., 2014; Wei et al., 2018).

Whether CCI captures causal understanding remains unclear. The causal and non-causal subsets may differ in how easily time reversal can be detected. For example, a video of a vase shattering may be easier to distinguish from its reversed version than a video of a car cruising. A higher RSI on the causal subset may therefore reflect easier reversal detection rather than causal understanding.

References:

Yuan et al. (2025). LikePhys: Evaluating Intuitive Physics Understanding in Video Diffusion Models via Likelihood Preference.

Pickup et al. (2014). Seeing the Arrow of Time. CVPR.

Wei et al. (2018). Learning and Using the Arrow of Time. CVPR.

Questions for authors

Could the authors report VLM classification accuracy alongside F1 and expand the human validation beyond 60 clearly causal or non-causal videos? Reporting results separately for videos with different classification difficulties, particularly ambiguous cases, would help establish labeling reliability across the full dataset.

Human judgments should be reliable, especially since participants can select “About the same.” The Kendall correlation of 0.667 is not particularly high, and Table 7 shows several disagreements. Could the authors analyze these cases, such as HunyuanVideo ranking third under human preference but sixth under YoCausal, and explain the differences?

Table 5 lists different prediction targets, including noise, velocity, and flow. Does the evaluation use each model’s original training loss or convert its output to a common noise-prediction loss? Although RSI compares only the sign of the forward–reverse loss difference, different loss formulations and timestep weightings could affect that sign. How are these differences handled?

How does CCI separate causal understanding from differences in reversal-detection difficulty? The causal and non-causal subsets may differ in how easily temporal direction can be identified from other cues.

Several models have similar RSI and CCI scores. Could the authors report confidence intervals and test whether differences between models are statistically significant? How stable are the rankings under resampling videos, diffusion timesteps, and noise?

6 comments
AuthorsOpenReview ↗

Response to Reviewer fFYN (Part 1/2)

We thank the reviewer for recognizing that reversal-based pairs need no generation or synthetic data and extend easily to diverse real-world scenes.

W1 — Novelty.
We respectfully clarify that LikePhys (Yuan et al., 2025), Pickup et al. (2014) and Wei et al. (2018) address intuitive physics and arrow-of-time detection, whereas we address causal cognition. Our novelty is threefold:

  1. Time reversal as a causal counterfactual. Reversal swaps cause and effect, so any real-world video yields a counterfactual at zero cost, enabling causality evaluation beyond simulated physics, including human and animal events.
  2. Disentangling causality from the arrow of time. Inspired by the infant experiment of Leslie & Keeble (1987), CCI contrasts causal and non-causal videos to isolate causal cognition.
  3. A new evaluation target: causal cognition in generative VDMs, read directly from the model's likelihood without generation or external judges.
QuestionMethodDataCollection
LikePhysIntuitive physicsLikelihood on physics violationsSimulated physics videosPhysics engine
Pickup / WeiArrow-of-time detectionClassify forward vs. reversedReversed real-world videosExisting video sets
OursCausal cognitionLikelihood on time-direction and causality violationsReversed real-world videosExisting video sets

Unlike LikePhys, we need no simulator, extending evaluation from simulated physics to real-world causal events. Unlike Pickup et al. and Wei et al., temporal direction is not our end goal: RSI measures it as a first level, and CCI goes beyond it to isolate causal cognition.

W2 / Q4 — Easier reversal detection in the causal subset?
The extra detectability in the causal subset is the causal signal itself, not a confound. We agree that a shattering vase is easier to recognize as reversed than a cruising car; that extra cue is the causal anomaly itself, which a model can exploit only if it expects cause before effect.

Since non-causal cues could also ease reversal detection, we verified that the two subsets are matched on them, so they cancel in CCI:

  • Motion. The optical-flow magnitude distributions are nearly identical (Cohen's dd = 0.057 < 0.2; Fig. 8a).
  • Duration and captions. Clip duration is fixed within each subset, and all prompts in a subset share the same captioning source and structure.
  • Scenes and objects. We labelled each video with a Places365 scene category and a COCO object supercategory; within every domain, the two subsets differ only slightly (Cramér's V≤0.28V \le 0.28, conventionally a small effect).

Model behaviour points the same way. If the causal subset were simply easier, models that detect reversal well (high RSI) would also score high on CCI, and the two metrics would rank models similarly. Instead, their rank correlation across the 13 models is low (Kendall's τ = 0.13).

Q1 — VLM labelling: accuracy, expanded human validation, and difficulty-stratified analysis.
CCI relies on a VLM to split each domain into causal and non-causal subsets, so we validate this split against human labels. We now report accuracy alongside F1, expand the human validation set, and stratify it by difficulty.

Expanded validation. We added 100 human-labelled videos to the 30 causal / 30 non-causal set, giving 80 / 80 (160 videos). Agreement is essentially unchanged:

Human-labelled setAccuracyF1Kendall's τ
30 / 30 (paper)80.77%82.76%0.761
80 / 80 (expanded)79.37%81.14%0.744

Difficulty-stratified analysis. Human annotators split the 160 videos into easy cases (causality clearly present or absent) and hard cases (ambiguous causality):

DifficultyAccuracyF1
Easy88.76%91.94%
Hard67.61%54.90%

The VLM agrees closely with humans on clear cases, indicating that its causality judgement is consistent with human judgement. We will add both tables to the revision.

(Continued in Part 2: Q2, Q3, Q5.)

AuthorsOpenReview ↗

Response to Reviewer fFYN (Part 2/2)

(Continued from Part 1: W1, W2/Q4, Q1.)

Q2 — Human correlation and HunyuanVideo.
τ = 0.667 is high and close to human–human agreement: 65 of the 78 model pairs (83%) are ordered the same way as by humans (pp < 0.001). Splitting the human raters into two random halves 1,000 times gives a human–human agreement of τ = 0.777 [0.615, 0.923], so YoCausal reaches ~86% of human–human agreement (0.667 / 0.777); achieving 80–90% of inter-human consistency indicates high reliability.
HunyuanVideo: YoCausal reads the model's prior on real videos, whereas the preference study judges generated videos, where visual fidelity also shapes perceived plausibility. HunyuanVideo's high visual quality likely lifts its human preference rank.

Q3 — Noise, velocity and flow losses.
We thank the reviewer for this insightful question. Each model is evaluated with its own native training loss, and the three formulations are essentially the same: each is a reparameterization of the noise (score) prediction. With xt=αtx0+σtϵx_t=\alpha_t x_0+\sigma_t\epsilon:

\text{Noise:}\quad & \mathcal{L}_\epsilon=\|\epsilon-\hat\epsilon\|^2 \\ \text{Velocity } (v=\alpha_t\epsilon-\sigma_t x_0):\quad & \mathcal{L}_v=\tfrac{1}{\alpha_t^2}\,\|\epsilon-\hat\epsilon\|^2 \\ \text{Flow } (u=\epsilon-x_0):\quad & \mathcal{L}_u=\tfrac{1}{(1-t)^2}\,\|\epsilon-\hat\epsilon\|^2 \end{aligned}$$ At a given timestep, the three losses differ only by a positive constant. Forward and reversed clips are evaluated at the same timestep with the same noise, so this constant scales both losses equally and does not change which is larger: the sign of $\mathcal{L}_r-\mathcal{L}_f$ is unchanged regardless of the loss formulation. In addition, RSI compares only the relative loss of forward and reversed clips under the same model and preprocessing, so model-side factors cancel by design, and CCI inherits this property. **Q5 — Confidence intervals.** We quantify the uncertainty of every score with a stratified bootstrap (2,000 resamples of videos within each subset, for all models). The intervals are reported in the Supplementary Material ("Uncertainty Estimates"). **Stability under noise and timestep sampling.** RSI averages each forward/reversed pair's denoising loss over $K$ = 10 sampled timesteps, with one Gaussian noise sample per timestep shared by both clips; CCI is RSI on the causal subset minus RSI on the non-causal subset. We ran two new experiments to measure the variability from noise and timestep sampling, on four models covering all three prediction targets: AnimateDiff-SD1.5 ($\epsilon$), CogVideoX-2B ($v$), Wan2.1-1.3B and LTXV-2B (flow). Everything else follows the paper's setting. | Model | RSI (%), 5 seeds | RSI (%), $K$ = 5 / 10 / 15 | CCI (%), 5 seeds | CCI (%), $K$ = 5 / 10 / 15 | | --- | --- | --- | --- | --- | | AnimateDiff-SD1.5 | 44.91 ± 0.54 | 44.37 / 45.69 / 45.75 | −5.29 ± 0.33 | −4.64 / −5.18 / −4.58 | | CogVideoX-2B | 46.70 ± 0.21 | 47.13 / 46.89 / 46.65 | −0.25 ± 1.25 | −1.24 / 0.31 / 1.28 | | Wan2.1-1.3B | 45.93 ± 0.30 | 46.42 / 45.51 / 47.19 | 5.10 ± 0.65 | 2.83 / 5.39 / 6.77 | | LTXV-2B | 56.75 ± 1.23 | 56.43 / 58.86 / 56.44 | −0.71 ± 1.52 | −5.08 / −0.16 / 0.72 | *Seed columns: mean ± std across 5 noise seeds at $K$ = 10.* Across noise seeds, the standard deviation is 0.30–1.23 points for RSI and 0.33–1.52 points for CCI, i.e., moderate variability. Going from $K$ = 10 to $K$ = 15 changes scores by at most 1.4 points, except LTXV-2B's RSI. Hence increasing $K$ beyond 10 brings little further change and $K$ = 10 is a sufficient and effective number of sampled timesteps. Since RSI and CCI are Monte-Carlo estimates of the expected denoising loss, averaging more noise seeds would make them more robust given sufficient compute. Each extra seed or timestep budget requires another full pass over the benchmark, so sweeping the larger models (5B–14B) would not finish within the rebuttal period on our allocated 24 GB RTX 4090 GPU. We will extend the experiment to them in the revision once larger GPUs are available. We thank the reviewer again and are glad to answer further within the remaining discussion time. If these responses address your concerns, we would be grateful if you would consider adjusting your score. Sincerely, The Authors

Dear Reviewer fFYN,

As the discussion period ends soon, we wanted to follow up. Our response addresses your concerns via (1) showing that CCI isolates causal cognition rather than reversal-detection ease: the causal and non-causal subsets are matched on motion, scenes and objects, and RSI and CCI rank models differently (Kendall's τ = 0.13); and (2) new reliability evidence: an expanded, difficulty-stratified VLM validation, a human–human agreement baseline (τ = 0.777), a proof that noise, velocity and flow losses keep the same sign, and stability across noise seeds, timesteps and resampling. Please let us know if anything remains unclear. Thank you!

Sincerely, The Authors

Reviewer_vXXTOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes YoCausal, a benchmark inspired by the Violation of Expectation (VoE) paradigm in cognitive science, to evaluate the causal cognition ability of Video Diffusion Models (VDMs). The core idea is to reverse the time of real videos as a counterfactual construction method, and to distinguish the model's sensitivity to temporal direction and its understanding of causal relationships through a two-stage evaluation. Specifically, in the first stage, the Reverse Surprise Index (RSI) is used to compare the denoising loss of the model on forward and backward videos, in order to measure the model's preference for temporal direction. In the second stage, the Causal Cognition Index (CCI) is introduced, and videos are divided into causal and non-causal subsets using a Visual Language Model (VLM). The RSI is then compared between the two types of samples, with the aim of reducing the interference of general temporal bias on causal evaluation. Experimental results show that the model's ability to perceive temporal direction does not necessarily mean that it truly understands the causal relationships in videos. There is still a significant gap between its causal cognition ability and that of humans.

Critical strengths

1.Any real-world video can be time-reversed at zero cost to generate counterfactual samples. By sharing content between forward and reverse sequences while only altering the temporal direction, this approach is closer to the real world than synthetic counterfactuals, while also facilitating subset expansion and cross-domain decomposition. 2. Transplant the expectancy violation paradigm from cognitive science into video diffusion models, using denoising loss as a proxy for "surprise" with variational inference grounded in "loss upper bound NLL." Fixing time steps and controlling noise levels equally ensures denoising difficulty, while RSI adopts a proportional formulation for robustness. 3. RSI measures the perception of the time arrow, while CCI attempts to eliminate statistical time bias by using the difference in RSI between causal and non-causal subsets. This two-tier design corresponds to Leslie & Keeble's infant paradigm and holds diagnostic value, with four thematic subsets also supporting domain decomposition.

Critical weaknesses
  1. YoCausal claims to support "significant breakthroughs" with "zero-cost utilization of any real-world video, of any scale, and with diverse scenes," but it fails to define scene granularity and de-duplication rules, does not report diversity statistics, and actually relies on four existing datasets. The term "zero-cost" is only applicable to constructing counterfactual samples through time reversal, thus making it impossible to effectively distinguish between "breakthroughs in real-world scene coverage" and "overestimates caused by relying on existing datasets and counting standards." Assuming that newly added subsets can be seamlessly integrated, even if the model evaluation itself is reliable, it may still overestimate scene diversity and weaken benchmark credibility due to homogeneity, subset imbalance, and lack of quality control.
  2. The "causal disentanglement" of CCI is not thorough, and the comparability of Dc/Dnc is insufficient. CCI = RSI(Dc) − RSI(Dnc) implies an assumption that the two groups are comparable in all dimensions except for causal significance; however, causal videos often exhibit more intense motion, more obvious contact, and stronger anomalies in reverse playback, while non-causal videos may also exhibit anomalies in reverse playback. VLM only judges "whether there is significant causal interaction," and cannot guarantee that the two groups match in terms of motion intensity, objects, scenes, duration, and textual descriptions. Therefore, CCI differences may still be mixed with "detectability of reverse playback anomalies" rather than pure causal understanding.
Questions for authors

None.

5 comments
AuthorsOpenReview ↗

Response to Reviewer vXXT

We thank the reviewer for recognizing the three pillars of our work: zero-cost real-world counterfactuals that stay close to reality and ease extension, a theoretical grounding in VoE and the NLL bound, and a two-level design that mirrors Leslie & Keeble's infant paradigm with real diagnostic value.

W1 — Scene diversity and "zero-cost".
To verify that our scene diversity is not inflated by redundant content, we propose a scene-similarity-based de-duplication procedure and count the unique scenes that remain. Each video is embedded with DINOv3 ViT-L/16 (semantic similarity) and SSCD (copy detection; Pizzi et al., CVPR 2022), averaged over 16 uniformly sampled frames. Following the SSCD paper, two videos with cosine similarity above 0.75 count as the same scene, and duplicates are removed greedily. As a reference, we apply the same rule to two lab-recorded benchmarks, Physics IQ and Phys101:

FeatureOursPhysics IQPhys101
DINOv31,232 → 1,132 (91.9%)396 → 59 (14.9%)9,113 → 98 (1.08%)
SSCD1,232 → 1,192 (96.8%)396 → 142 (35.9%)9,113 → 165 (1.81%)
Unique under both1,232 → 1,130 (91.7%)396 → 58 (14.6%)9,113 → 73 (0.80%)

Our scene diversity is almost not inflated. Under the same de-duplication rule, the other real-world benchmarks retain far fewer unique scenes, even though Phys101 has over 7 times more videos than ours. Since de-duplication may add a small compute cost, we will write "low-cost" instead of "zero-cost" and update Table 1.

De-duplication rules out scene homogeneity, and equal-weight averaging across domains prevents subset imbalance. For quality control, adding a VLM-based quality filter will be the future improvement.

W2 — Comparability of the causal and non-causal subsets.

CCI is valid when the causal and non-causal subsets differ mainly in causal content. We verify that the two subsets are matched on every dimension the reviewer lists:

  • Motion intensity. The two subsets are nearly identical in optical-flow magnitude distribution (Cohen's dd = 0.057 < 0.2, Fig. 8a).
  • Duration and captions. Clip duration is fixed within each subset, and all prompts in a subset share the same captioning source and structure. Since CCI is computed within each subset, both factors are matched by construction.
  • Scenes and objects. We labelled each video with a Places365 L2 scene category and a COCO object supercategory:
SubsetAnimalGeneralHumanPhysics
Scene VV0.060.250.280.12
Object VV0.110.190.230.25

All values are below 0.3, which is conventionally a small effect and barely account for CCI.

We thank the reviewer again, and are glad to answer further within the remaining discussion time. If these responses address your concerns, we would be grateful if you would consider adjusting your score.

Sincerely, The Authors

Dear Reviewer vXXT,

As the discussion period ends soon, we wanted to follow up. Our response addresses your concerns via (1) a scene-diversity check: under DINOv3 + SSCD de-duplication, 91.7% of our videos remain unique scenes, versus 14.6% for Physics IQ and 0.80% for Phys101, and we will revise "zero-cost" to "low-cost"; and (2) evidence that the causal and non-causal subsets are comparable: they are matched on motion intensity (Cohen's d = 0.057), share duration and captioning by construction, and differ only slightly in scenes and objects (Cramér's V < 0.3 in every domain). Please let us know if anything remains unclear. Thank you!

Sincerely, The Authors

Dear Reviewer,

The discussion period is nearing its end, and we haven’t yet seen a reply from you. We encourage you to respond to the authors’ comments, especially if you have further questions about points that may affect your assessment.

Thank you for taking the time to engage in the discussion.

Best,

Your AC

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

We thank all reviewers for their constructive feedback. We are encouraged that the reviewers recognize:

  • Real-world counterfactuals at scale (vXXT, fFYN, uJxQ). Time reversal turns any real video into a counterfactual without simulation, generation or annotation, so the benchmark avoids the sim-to-real gap and extends easily to new domains.
  • A cognitively grounded two-level design (vXXT, uJxQ). The benchmark adapts the infant Violation-of-Expectation paradigm, and its two-level metrics separate causal sensitivity (CCI) from arrow-of-time perception (RSI).
  • Complementary validation (uJxQ): human preference, alternative VLM judges, and a null-prompt ablation.

New in this rebuttal:

AnalysisResultReviewer
Scene de-duplication91.7% of our videos are distinct scenes (vs. 14.6% for Physics IQ and 0.80% for Phys101)vXXT
Scene/object matching of causal vs. non-causal subsetsCramér's V≤0.28V \le 0.28 (small)all
VLM labelling: expanded and difficulty-stratified validation80/80 set: accuracy 79.4%, F1 81.1%; easy 88.8% vs. hard 67.6% accuracyfFYN
RSI + CCI vs. human preferenceR2R^2 0.56 → 0.79 (adjusted 0.52 → 0.75, pp = 0.007)uJxQ
Human split-half ceilingτ = 0.777; ours 0.667 (86% of ceiling)fFYN
Human preference vs. aesthetic qualityτ = 0.179 (pp = 0.435, n.s.), far below τ = 0.667 with YoCausal, so raters judged causality rather than visual qualityfFYN
Bootstrap confidence intervalsIn the Supplementary MaterialfFYN, uJxQ
Noise-seed (5 seeds) and timestep-budget (KK = 5/10/15) stabilityModerate variability (seed std ≤ 1.52 points); KK = 10 → 15 changes CCI within seed variability, so KK = 10 is sufficientfFYN, uJxQ

Sincerely, The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.