- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
CausalMotion is a training-free pipeline that combines VLM-planned visual states with object trajectories to guide a pretrained LTX-Video model. It generates keyframes through iterative reasoning and GPT-image-1.5 editing, predicts sparse image-plane object states with Qwen2.5-VL-72B, interpolates trajectories, and aligns keyframes to them using bounding-box overlap. During sampling, it combines multi-keyframe conditioning with localized latent updates using neighboring keyframe appearance anchors and warped preceding-frame features.
Critical strengths
-
The combination of explicit scene-state anchors and object-motion guidance is a useful, interpretable way to add planning to an existing video generator without retraining (Sections 3.1–3.5). The method addresses complementary failure modes: missing intermediate visual states and inconsistent motion between those states.
-
The reported end-to-end gains are encouraging across several physical domains: Table 1 improves over both text-conditioned and image-conditioned LTX-Video, and Table 2 suggests that perceptual quality is preserved. The paper also provides component ablations, alignment diagnostics, a runtime breakdown, and failure cases (Table 3; Appendices A, C, and F), which help characterize the practical behavior of the pipeline.
Critical weaknesses
-
Limited methodological novelty beyond combining established planning and guidance components. VChain already uses multimodal reasoning to construct keyframe-based visual plans, while VLIPP uses VLM-predicted motion trajectories to guide video generation. CausalMotion's main distinction is their combination through IoU-based temporal alignment and localized latent blending (Secs. 3.3–3.4). Avoiding inference-time tuning, unlike VChain, is a useful practical difference, but the paper does not convincingly establish what new capability or generalizable insight the proposed coupling contributes beyond these ingredients. In Table 3, removing trajectory planning or latent trajectory guidance changes PhyGenBench from 0.654 to 0.642 or 0.645, respectively. These modest increments, without a clearly matched comparison isolating the coupling, leave the methodological contribution insufficiently distinguished from an integration of existing ideas.
-
The proposed guidance does not directly address the reliability of the physical reasoning on which it depends. The introduction motivates the work through the limitations of statistical correlations, but the keyframe consequences and sparse motion states are still inferred by pretrained models. The explicit state in Eq. (2) consists of image-plane boxes and velocities; Eq. (3) interpolates these predictions, Eq. (4) aligns box overlap, and Eqs. (5)–(6) encourage spatial and appearance consistency. These operations can make a video follow a plan more faithfully, but do not establish that the plan or the rendered interactions are physically valid. For example, consistent boxes alone do not determine whether a collision response or material transformation is correct. The reported benchmark gains support improved plausibility, yet do not isolate improved physical reasoning from stronger conditioning on plausible-looking keyframes. This limits the paper's claimed advance on the underlying physical-consistency problem.
Questions for authors
-
What is the specific methodological advance of the joint keyframe–trajectory mechanism beyond combining visual-state planning and motion guidance? Can the existing ablations isolate the benefit of the proposed coupling using the same generated keyframes and sampling seeds—for example, comparing the full method with keyframe conditioning plus a simple trajectory-guidance variant—and report paired per-prompt gains? Please distinguish the contribution of temporal alignment from that of latent blending.
-
What evidence distinguishes better physical state-transition reasoning from better adherence to the supplied keyframes and trajectories? On the existing evaluation examples, can you separate errors in the planned states/trajectories from errors introduced during rendering, particularly for collisions or material changes, and show whether the pipeline detects or corrects physically incorrect plans? This would clarify which part of the physical-consistency problem the method actually resolves.
10 comments
Response to Reviewer VfBk (2/3)
W1.2: These modest increments, without a clearly matched comparison isolating the coupling, leave the methodological contribution insufficiently distinguished from an integration of existing ideas.
Q1.2: Can the existing ablations isolate the benefit of the proposed coupling using the same generated keyframes and sampling seeds—for example, comparing the full method with keyframe conditioning plus a simple trajectory-guidance variant—and report paired per-prompt gains? Please distinguish the contribution of temporal alignment from that of latent blending.
We thank the reviewer for the constructive suggestions. The key benefit of our joint mechanism is to establish temporal correspondence between visual key states and object motion, and use this correspondence to guide continuous video generation. The key benefit of our joint mechanism is to establish temporal correspondence between visual key states and object motion, and use this correspondence to guide continuous video generation.
Ablation Study
We compare full CausalMotion with three variants:
- Keyframe-only: Uses the same generated keyframes with uniform temporal placement, without trajectory planning, alignment, or latent guidance.
- w/o Latent Trajectory Guidance: Retains trajectory-aware keyframe alignment but disables localized latent updates.
- w/ Simplified Guidance: Uses uniformly placed keyframes and linearly interpolated bounding-box trajectories instead of VLM-predicted trajectories.
All variants share the evaluation prompts, generated keyframes, and sampling seeds, but differ in temporal placement and trajectory guidance.
Table A. Ablation results comparing temporal alignment and trajectory-guided latent updates.
| Method | PhyGenBench ↑ | VBench ↑ | VLM Temp. ↑ | VLM Overall ↑ |
|---|---|---|---|---|
| Keyframe-only (Uniform Placement) | 0.642 ± 0.010 | 82.44 | 3.65 | 3.37 |
| w/o Latent Trajectory Guidance | 0.645 ± 0.006 | 82.46 | 3.90 | 3.55 |
| w/ Simplified Guidance | 0.649 ± 0.006 | 81.62 | 3.80 | 3.70 |
| Full CausalMotion | 0.662 ± 0.010 | 82.52 | 4.65 | 4.37 |
Distinguishing Temporal Alignment and Latent Blending
Temporal Alignment. Appendix B.2 (Fig. 9) provides a direct qualitative comparison between uniform and trajectory-aligned keyframe placement, using identical keyframes, prompts, and random seeds, with latent trajectory guidance disabled. Under uniform placement, the object contacts the pillow and subsequently moves upward again. In contrast, trajectory-aware alignment produces a coherent falling–contact–resting sequence.
Latent Blending. With trajectory-aware alignment retained, adding localized latent guidance improves the PhyGenBench score from 0.645 to 0.662 and VLM temporal consistency from 3.90 to 4.65. Appendix B.3 (Fig. 10) further illustrates abrupt appearance and geometric changes that may arise under native multi-keyframe conditioning without latent trajectory guidance.
Effectiveness of the Joint Mechanism
As shown in Table A, full CausalMotion achieves a higher PhyGenBench and VLM overall score. These results demonstrate the practical benefit of the complete coupling mechanism over simplified 2D interpolation.
We further compute per-prompt paired gains using the mean of three VLM-as-Judge dimensions across three sampling seeds:
\left[S_{\mathrm{Full}}(p,s)-S_{\mathrm{Simplified}}(p,s)\right].$$ **Table B. Exploratory prompt-level paired comparison between full CausalMotion and Simplified Guidance.** | Evaluation Set | Improved | Tied | Decreased | Mean Paired Gain | | --------------------------- | -------- | ---- | --------- | ---------------- | | VChain-aligned prompts (20) | 16 | 2 | 2 | +0.7 | | PhyGenBench prompts (160) | 134 | 19 | 7 | +0.793 | **Table C. Representative per-prompt paired VLM-as-Judge scores for Simplified Guidance and full CausalMotion, averaged across three sampling seeds.** | Physical Scenario | Simplified Guidance | Full CausalMotion | Paired Gain | | -------------------------------- | ------------------- | ----------------- | ----------- | | Blowing out birthday candles | 3.667 | **4.889** | **+1.222** | | An egg falling onto a pillow | 3.222 | **3.778** | **+0.556** | | Ice cream melting under sunlight | 4.000 | **4.556** | **+0.556** | Overall, temporal alignment and latent blending jointly improve state-transition consistency through training-free guidance, with complementary contributions to temporal correspondence and intermediate-frame continuity.Response to Reviewer VfBk (3/3)
W2: The proposed guidance does not directly address the reliability of the physical reasoning on which it depends. The introduction motivates the work through the limitations of statistical correlations, but the keyframe consequences and sparse motion states are still inferred by pretrained models. The explicit state in Eq. (2) consists of image-plane boxes and velocities; Eq. (3) interpolates these predictions, Eq. (4) aligns box overlap, and Eqs. (5)–(6) encourage spatial and appearance consistency. These operations can make a video follow a plan more faithfully, but do not establish that the plan or the rendered interactions are physically valid. For example, consistent boxes alone do not determine whether a collision response or material transformation is correct. The reported benchmark gains support improved plausibility, yet do not isolate improved physical reasoning from stronger conditioning on plausible-looking keyframes. This limits the paper's claimed advance on the underlying physical-consistency problem.
Q2: What evidence distinguishes better physical state-transition reasoning from better adherence to the supplied keyframes and trajectories? On the existing evaluation examples, can you separate errors in the planned states/trajectories from errors introduced during rendering, particularly for collisions or material changes, and show whether the pipeline detects or corrects physically incorrect plans? This would clarify which part of the physical-consistency problem the method actually resolves.
We thank the reviewer for highlighting the distinction between the physical correctness of a plan and the generator's ability to follow it.
CausalMotion primarily improves the realization of structured physical priors in video generation, rather than independently verifying the physical correctness of VLM predictions. Our framework separates this process into two stages:
- Physical Planning (Secs. 3.1–3.2). A pretrained VLM infers the expected outcome, intermediate key states, and object motion trends, producing structured physical priors represented by keyframes and trajectories.
- Physical Realization (Secs. 3.3–3.4). Temporal alignment establishes correspondence between these representations, and trajectory-guided latent updates encourage the generator to realize the planned motion and state transitions consistently.
Our ablation results provide evidence for the complementary contributions of these two stages. In Table 3 of the revised paper, removing keyframe reasoning reduces the PhyGenBench score from 0.662 to 0.471, while disabling latent trajectory guidance reduces it to 0.645. Figure 4 further illustrates how removing these components affects collision dynamics and material transformations. These results demonstrate that structured physical priors and their effective integration into video generation jointly improve the physical plausibility of the final videos. However, these end-to-end evaluations do not independently establish the physical correctness of the VLM-generated plans or fully distinguish planning errors from rendering errors.
Limitations and Future Work. We acknowledge that the reliability of VLM-generated physical plans remains an important limitation. While our method improves the temporal and spatial consistency of planned state transitions, it does not guarantee that the underlying plans satisfy physical laws. In particular, inaccurate predictions of collision responses or material transformations may propagate into the generated videos. Our current framework focuses on realizing the inferred physical priors rather than detecting or automatically correcting invalid plans. We appreciate the reviewer's suggestion and will explore explicit physical-plan verification, stage-wise error attribution, and correction mechanisms in future work.
In summary, CausalMotion addresses the gap between structured physical planning and video synthesis by enabling more coherent realization of predicted physical states and trajectories. Our results demonstrate the effectiveness of this training-free integration in improving physical plausibility and temporal consistency, while more reliable physical-plan verification remains an important direction for future research.
We sincerely thank the reviewer again for the insightful feedback, which helps us clarify the scope of our contribution and identify meaningful directions for future improvement.
Best regards,
Authors
Response to Reviewer VfBk (1/3)
We sincerely thank the reviewer for the constructive comments and thoughtful questions. We appreciate the opportunity to clarify our methodological contribution, provide further evidence for the joint keyframe–trajectory guidance mechanism, and explain the scope of physical consistency addressed by CausalMotion.
We agree that both keyframe inference and trajectory guidance have been studied previously. The contribution of CausalMotion is not in proposing these individual techniques for the first time, but rather in explicitly aligning the two representations in terms of time and jointly applying them to video generation without the need for model parameter updates. Specifically, our approach differs from related methods in the following ways:
The key insight is that visual state planning and motion prediction provide complementary constraints, but require temporal correspondence to jointly guide generation. Our IoU-based alignment (Eq. 4) establishes this correspondence, while localized latent blending (Eqs. 5–6) propagates appearance and motion information between key states.