acceptodds
Under review as a conference paper at ICLR 2027

CausalMotion: Structured Physical Reasoning as Keyframe and Trajectory Guidance for Training-Free Video Generation

Abstract

Recent advances in diffusion-based video generation have significantly improved visual quality and short-term temporal coherence. However, existing video diffusion models, while effective at modeling local structures and motion, often struggle with long-horizon physical inference, resulting in dynamics that lack physical consistency and causal plausibility. We therefore propose **CausalMotion**, a training-free framework that injects explicit physical reasoning into video generation through structured intermediate representations. Our key idea is to decouple reasoning from generation by leveraging a vision-language model to translate the text prompt into a sequence of causally consistent scene states represented by keyframes, together with object-centric motion trajectories. These representations provide complementary spatial and temporal guidance, which we align and integrate as soft constraints into a pretrained video diffusion model during inference. By explicitly planning object states and dynamics before generation, CausalMotion improves physical plausibility and temporal coherence without additional training or supervision. Extensive experiments demonstrate consistent improvements over existing baselines, particularly in dynamics-intensive scenarios, while maintaining high perceptual video quality.

Primary area:
generative models
Keywords:
Physics-Consistent Video Generation

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.50 over 4 reviews, above 68% of the others.
Reviewer_VfBkOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

CausalMotion is a training-free pipeline that combines VLM-planned visual states with object trajectories to guide a pretrained LTX-Video model. It generates keyframes through iterative reasoning and GPT-image-1.5 editing, predicts sparse image-plane object states with Qwen2.5-VL-72B, interpolates trajectories, and aligns keyframes to them using bounding-box overlap. During sampling, it combines multi-keyframe conditioning with localized latent updates using neighboring keyframe appearance anchors and warped preceding-frame features.

Critical strengths
  1. The combination of explicit scene-state anchors and object-motion guidance is a useful, interpretable way to add planning to an existing video generator without retraining (Sections 3.1–3.5). The method addresses complementary failure modes: missing intermediate visual states and inconsistent motion between those states.

  2. The reported end-to-end gains are encouraging across several physical domains: Table 1 improves over both text-conditioned and image-conditioned LTX-Video, and Table 2 suggests that perceptual quality is preserved. The paper also provides component ablations, alignment diagnostics, a runtime breakdown, and failure cases (Table 3; Appendices A, C, and F), which help characterize the practical behavior of the pipeline.

Critical weaknesses
  1. Limited methodological novelty beyond combining established planning and guidance components. VChain already uses multimodal reasoning to construct keyframe-based visual plans, while VLIPP uses VLM-predicted motion trajectories to guide video generation. CausalMotion's main distinction is their combination through IoU-based temporal alignment and localized latent blending (Secs. 3.3–3.4). Avoiding inference-time tuning, unlike VChain, is a useful practical difference, but the paper does not convincingly establish what new capability or generalizable insight the proposed coupling contributes beyond these ingredients. In Table 3, removing trajectory planning or latent trajectory guidance changes PhyGenBench from 0.654 to 0.642 or 0.645, respectively. These modest increments, without a clearly matched comparison isolating the coupling, leave the methodological contribution insufficiently distinguished from an integration of existing ideas.

  2. The proposed guidance does not directly address the reliability of the physical reasoning on which it depends. The introduction motivates the work through the limitations of statistical correlations, but the keyframe consequences and sparse motion states are still inferred by pretrained models. The explicit state in Eq. (2) consists of image-plane boxes and velocities; Eq. (3) interpolates these predictions, Eq. (4) aligns box overlap, and Eqs. (5)–(6) encourage spatial and appearance consistency. These operations can make a video follow a plan more faithfully, but do not establish that the plan or the rendered interactions are physically valid. For example, consistent boxes alone do not determine whether a collision response or material transformation is correct. The reported benchmark gains support improved plausibility, yet do not isolate improved physical reasoning from stronger conditioning on plausible-looking keyframes. This limits the paper's claimed advance on the underlying physical-consistency problem.

Questions for authors
  1. What is the specific methodological advance of the joint keyframe–trajectory mechanism beyond combining visual-state planning and motion guidance? Can the existing ablations isolate the benefit of the proposed coupling using the same generated keyframes and sampling seeds—for example, comparing the full method with keyframe conditioning plus a simple trajectory-guidance variant—and report paired per-prompt gains? Please distinguish the contribution of temporal alignment from that of latent blending.

  2. What evidence distinguishes better physical state-transition reasoning from better adherence to the supplied keyframes and trajectories? On the existing evaluation examples, can you separate errors in the planned states/trajectories from errors introduced during rendering, particularly for collisions or material changes, and show whether the pipeline detects or corrects physically incorrect plans? This would clarify which part of the physical-consistency problem the method actually resolves.

10 comments
AuthorsOpenReview ↗

Response to Reviewer VfBk (1/3)

We sincerely thank the reviewer for the constructive comments and thoughtful questions. We appreciate the opportunity to clarify our methodological contribution, provide further evidence for the joint keyframe–trajectory guidance mechanism, and explain the scope of physical consistency addressed by CausalMotion.


W1.1: Limited methodological novelty beyond combining established planning and guidance components.

Q1.1: What is the specific methodological advance of the joint keyframe–trajectory mechanism beyond combining visual-state planning and motion guidance?

We agree that both keyframe inference and trajectory guidance have been studied previously. The contribution of CausalMotion is not in proposing these individual techniques for the first time, but rather in explicitly aligning the two representations in terms of time and jointly applying them to video generation without the need for model parameter updates. Specifically, our approach differs from related methods in the following ways:

  • Compared with VChain, which uses visual reasoning to generate keyframes and incorporates them through sparse inference-time tuning, CausalMotion requires no parameter updates. It additionally predicts object-centric trajectories, determines keyframe conditioning timestamps through IoU-based temporal alignment, and applies localized trajectory-guided latent updates during denoising.
  • Compared with VLIPP, which uses VLM-predicted coarse motion priors for video generation, CausalMotion introduces explicit visual representations of intermediate physical states. Temporal alignment connects these keyframes with predicted motion trajectories, while latent guidance combines neighboring keyframe appearance anchors with warped previous-frame features to encourage coherent state transitions.
  • Compared with PhyRPR, CoECT, and DeforM, which explore physical reasoning and motion planning, event-level causal representations, and localized spatiotemporal guidance, respectively, CausalMotion specifically focuses on establishing explicit correspondence between key states and object-centric trajectories and integrating both representations through training-free latent guidance.

The key insight is that visual state planning and motion prediction provide complementary constraints, but require temporal correspondence to jointly guide generation. Our IoU-based alignment (Eq. 4) establishes this correspondence, while localized latent blending (Eqs. 5–6) propagates appearance and motion information between key states.

AuthorsOpenReview ↗

Response to Reviewer VfBk (2/3)

W1.2: These modest increments, without a clearly matched comparison isolating the coupling, leave the methodological contribution insufficiently distinguished from an integration of existing ideas.

Q1.2: Can the existing ablations isolate the benefit of the proposed coupling using the same generated keyframes and sampling seeds—for example, comparing the full method with keyframe conditioning plus a simple trajectory-guidance variant—and report paired per-prompt gains? Please distinguish the contribution of temporal alignment from that of latent blending.

We thank the reviewer for the constructive suggestions. The key benefit of our joint mechanism is to establish temporal correspondence between visual key states and object motion, and use this correspondence to guide continuous video generation. The key benefit of our joint mechanism is to establish temporal correspondence between visual key states and object motion, and use this correspondence to guide continuous video generation.

Ablation Study

We compare full CausalMotion with three variants:

  • Keyframe-only: Uses the same generated keyframes with uniform temporal placement, without trajectory planning, alignment, or latent guidance.
  • w/o Latent Trajectory Guidance: Retains trajectory-aware keyframe alignment but disables localized latent updates.
  • w/ Simplified Guidance: Uses uniformly placed keyframes and linearly interpolated bounding-box trajectories instead of VLM-predicted trajectories.

All variants share the evaluation prompts, generated keyframes, and sampling seeds, but differ in temporal placement and trajectory guidance.

Table A. Ablation results comparing temporal alignment and trajectory-guided latent updates.

MethodPhyGenBench ↑VBench ↑VLM Temp. ↑VLM Overall ↑
Keyframe-only (Uniform Placement)0.642 ± 0.01082.443.653.37
w/o Latent Trajectory Guidance0.645 ± 0.00682.463.903.55
w/ Simplified Guidance0.649 ± 0.00681.623.803.70
Full CausalMotion0.662 ± 0.01082.524.654.37

Distinguishing Temporal Alignment and Latent Blending

Temporal Alignment. Appendix B.2 (Fig. 9) provides a direct qualitative comparison between uniform and trajectory-aligned keyframe placement, using identical keyframes, prompts, and random seeds, with latent trajectory guidance disabled. Under uniform placement, the object contacts the pillow and subsequently moves upward again. In contrast, trajectory-aware alignment produces a coherent falling–contact–resting sequence.

Latent Blending. With trajectory-aware alignment retained, adding localized latent guidance improves the PhyGenBench score from 0.645 to 0.662 and VLM temporal consistency from 3.90 to 4.65. Appendix B.3 (Fig. 10) further illustrates abrupt appearance and geometric changes that may arise under native multi-keyframe conditioning without latent trajectory guidance.

Effectiveness of the Joint Mechanism

As shown in Table A, full CausalMotion achieves a higher PhyGenBench and VLM overall score. These results demonstrate the practical benefit of the complete coupling mechanism over simplified 2D interpolation.

We further compute per-prompt paired gains using the mean of three VLM-as-Judge dimensions across three sampling seeds:

\left[S_{\mathrm{Full}}(p,s)-S_{\mathrm{Simplified}}(p,s)\right].$$ **Table B. Exploratory prompt-level paired comparison between full CausalMotion and Simplified Guidance.** | Evaluation Set | Improved | Tied | Decreased | Mean Paired Gain | | --------------------------- | -------- | ---- | --------- | ---------------- | | VChain-aligned prompts (20) | 16 | 2 | 2 | +0.7 | | PhyGenBench prompts (160) | 134 | 19 | 7 | +0.793 | **Table C. Representative per-prompt paired VLM-as-Judge scores for Simplified Guidance and full CausalMotion, averaged across three sampling seeds.** | Physical Scenario | Simplified Guidance | Full CausalMotion | Paired Gain | | -------------------------------- | ------------------- | ----------------- | ----------- | | Blowing out birthday candles | 3.667 | **4.889** | **+1.222** | | An egg falling onto a pillow | 3.222 | **3.778** | **+0.556** | | Ice cream melting under sunlight | 4.000 | **4.556** | **+0.556** | Overall, temporal alignment and latent blending jointly improve state-transition consistency through training-free guidance, with complementary contributions to temporal correspondence and intermediate-frame continuity.
AuthorsOpenReview ↗

Response to Reviewer VfBk (3/3)

W2: The proposed guidance does not directly address the reliability of the physical reasoning on which it depends. The introduction motivates the work through the limitations of statistical correlations, but the keyframe consequences and sparse motion states are still inferred by pretrained models. The explicit state in Eq. (2) consists of image-plane boxes and velocities; Eq. (3) interpolates these predictions, Eq. (4) aligns box overlap, and Eqs. (5)–(6) encourage spatial and appearance consistency. These operations can make a video follow a plan more faithfully, but do not establish that the plan or the rendered interactions are physically valid. For example, consistent boxes alone do not determine whether a collision response or material transformation is correct. The reported benchmark gains support improved plausibility, yet do not isolate improved physical reasoning from stronger conditioning on plausible-looking keyframes. This limits the paper's claimed advance on the underlying physical-consistency problem.

Q2: What evidence distinguishes better physical state-transition reasoning from better adherence to the supplied keyframes and trajectories? On the existing evaluation examples, can you separate errors in the planned states/trajectories from errors introduced during rendering, particularly for collisions or material changes, and show whether the pipeline detects or corrects physically incorrect plans? This would clarify which part of the physical-consistency problem the method actually resolves.

We thank the reviewer for highlighting the distinction between the physical correctness of a plan and the generator's ability to follow it.

CausalMotion primarily improves the realization of structured physical priors in video generation, rather than independently verifying the physical correctness of VLM predictions. Our framework separates this process into two stages:

  1. Physical Planning (Secs. 3.1–3.2). A pretrained VLM infers the expected outcome, intermediate key states, and object motion trends, producing structured physical priors represented by keyframes and trajectories.
  2. Physical Realization (Secs. 3.3–3.4). Temporal alignment establishes correspondence between these representations, and trajectory-guided latent updates encourage the generator to realize the planned motion and state transitions consistently.

Our ablation results provide evidence for the complementary contributions of these two stages. In Table 3 of the revised paper, removing keyframe reasoning reduces the PhyGenBench score from 0.662 to 0.471, while disabling latent trajectory guidance reduces it to 0.645. Figure 4 further illustrates how removing these components affects collision dynamics and material transformations. These results demonstrate that structured physical priors and their effective integration into video generation jointly improve the physical plausibility of the final videos. However, these end-to-end evaluations do not independently establish the physical correctness of the VLM-generated plans or fully distinguish planning errors from rendering errors.

Limitations and Future Work. We acknowledge that the reliability of VLM-generated physical plans remains an important limitation. While our method improves the temporal and spatial consistency of planned state transitions, it does not guarantee that the underlying plans satisfy physical laws. In particular, inaccurate predictions of collision responses or material transformations may propagate into the generated videos. Our current framework focuses on realizing the inferred physical priors rather than detecting or automatically correcting invalid plans. We appreciate the reviewer's suggestion and will explore explicit physical-plan verification, stage-wise error attribution, and correction mechanisms in future work.

In summary, CausalMotion addresses the gap between structured physical planning and video synthesis by enabling more coherent realization of predicted physical states and trajectories. Our results demonstrate the effectiveness of this training-free integration in improving physical plausibility and temporal consistency, while more reliable physical-plan verification remains an important direction for future research.


We sincerely thank the reviewer again for the insightful feedback, which helps us clarify the scope of our contribution and identify meaningful directions for future improvement.

Best regards,

Authors

Reviewer_NbMMOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes CausalMotion, a training-free pipeline for making text-to-video outputs more physically plausible. Given a prompt, a VLM first reasons about the "final consequence" of the scene and then autoregressively writes captions for a chain of key states, which are rendered into keyframes with GPT-Image-1.5. A grounding model locates the key objects in the first keyframe, and the VLM (Qwen2.5-VL-72B) predicts sparse bounding-box states (position, size, velocity) at 13 timesteps, which are interpolated to 121 frames with either linear or constant-acceleration rules. Keyframes are aligned to the trajectory by greedy IoU matching, and the aligned keyframes are fed to LTX-Video as conditioning frames. During denoising, the latent inside each trajectory box is softly blended with a reference built from neighboring keyframe latents and a warped copy of the previous frame. On PhyGenBench the method scores 0.65 average (vs. 0.60 for VLIPP), and it also reports VBench and a Gemini-as-judge evaluation plus ablations of the three components.

Critical strengths
  • Sensible, clearly explained pipeline. Separating "what states should happen" (keyframes) from "where objects go" (trajectories) and then aligning the two is intuitive, and Figures 1–2 and the appendix pseudocode make the method easy to follow.
  • Training-free and modular. Every component is a drop-in off-the-shelf model, so the idea is easy to reproduce and to transfer to other backbones.
  • Reasonably broad evaluation. PhyGenBench, VBench, a VLM judge, ablations per component, a runtime breakdown, and an honest failure-case section (static appearance changes).
Critical weaknesses
  • Limited novelty over very close prior work. VLIPP (VLM-predicted trajectories as inference-time guidance), VChain (VLM keyframes as guidance), PhyRPR, CoECT and DeforM all do pieces of this. The paper's stated contribution is "we combine them," but there is no head-to-head comparison that isolates what the combination adds. VChain only appears in the VBench column of Table 2 and is absent from PhyGenBench and the VLM-judge rows; VLIPP is absent from Table 2 entirely.
  • The gains are small and have no error bars. The main-table improvement over VLIPP is 0.05, and the ablations move the average by 0.01 (0.654 → 0.642 / 0.645). With a GPT-Image-1.5 keyframe generator and a 72B VLM in the loop, this is well within the noise one would expect from sampling variance. No seeds, no confidence intervals, no number of prompts per category.
  • Most of the gain seems to come from keyframes, not the proposed trajectory machinery. Removing keyframe reasoning drops the score to 0.471, while removing trajectory planning or latent guidance barely changes it. The paper's novel parts (Sections 3.2–3.4) are therefore weakly supported. A stronger baseline is needed: LTX-Video conditioned on the same GPT-Image-1.5 keyframes at uniformly spaced indices, with no trajectory at all. The "w/o trajectory physical state" row may already be this, but it is not described precisely enough to tell.
  • Unfair comparison. CausalMotion uses GPT-Image-1.5 (a closed, state-of-the-art image model) to produce every keyframe, so part of the "physical plausibility" is simply inherited from that model, and the output is effectively an interpolation between high-quality stills. Baselines like LTX-Video-T2V get the text prompt only. The VLM-judge semantic-alignment jump (1.68 → 4.40) looks like a keyframe-quality effect, not a video-dynamics effect.
  • Key hyperparameters are undefined. The weights in Eq. 5, the TriWeight function, which denoising steps are "selected," the box expansion factor ρ, and the schedule for α are never given. The latent update (Eq. 6) is the main technical piece and cannot be reproduced from the text.
  • Cost. ~448 s per 4-second clip, with 270 s spent in paid API calls, is roughly two orders of magnitude slower than the baselines. This is acknowledged but not weighed against the 0.05 gain.
Questions for authors
  1. Can you report PhyGenBench and VLM-judge scores for VChain and VLIPP under the same backbone, and VBench for VLIPP, so all methods appear in both tables?
  2. Please run the full model and the ablations with at least 3 seeds and report mean ± std. Are the 0.01 ablation differences significant?
  3. What exactly does "w/o trajectory physical state" do? Is it LTX-Video + the same keyframes at uniform indices? If not, please add that baseline.
  4. How much of the gain survives if GPT-Image-1.5 is replaced by an open image model (e.g., FLUX or SDXL)? This would show whether the method or the keyframe generator is doing the work.
  5. Please specify the Eq. 5 weights, the selected guidance steps, ρ, and the α schedule.
  6. Since the VLM-judge prompt and the trajectory-planning VLM are both from the same family of instruction-tuned models, did you check for judge bias (e.g., by also using a second judge or human ratings on a subset)?
8 comments
AuthorsOpenReview ↗

Response to Reviewer NbMM (1/7)

We thank the reviewer for the detailed assessment and constructive suggestions. We have expanded our comparisons with closely related methods, conducted multi-seed evaluations and additional ablations, examined alternative keyframe generators, and clarified the implementation details and computational costs. We address each concern below.


W1: Limited novelty over very close prior work. VLIPP (VLM-predicted trajectories as inference-time guidance), VChain (VLM keyframes as guidance), PhyRPR, CoECT and DeforM all do pieces of this. The paper's stated contribution is "we combine them," but there is no head-to-head comparison that isolates what the combination adds. VChain only appears in the VBench column of Table 2 and is absent from PhyGenBench and the VLM-judge rows; VLIPP is absent from Table 2 entirely.

Q1: Can you report PhyGenBench and VLM-judge scores for VChain and VLIPP under the same backbone, and VBench for VLIPP, so all methods appear in both tables?

We appreciate the reviewer’s thoughtful comments on the novelty of our method and its relationship to prior work. We provide additional quantitative comparisons and clarify the distinctive contributions of CausalMotion below.

The main contribution of CausalMotion is the training-free integration of causally reasoned keyframes and object-centric trajectories through explicit temporal alignment and localized latent-space guidance. Rather than introducing keyframe reasoning or trajectory prediction individually, our approach establishes their temporal correspondence and jointly uses them to guide video generation without updating the pretrained model parameters.

Differences from related methods.

  • Compared with VChain: VChain generates keyframes through visual reasoning and incorporates them using sparse inference-time tuning. CausalMotion instead predicts object-centric trajectories, assigns keyframe timestamps through IoU-based temporal alignment, and applies localized trajectory constraints during diffusion sampling, without parameter updates.
  • Compared with VLIPP: VLIPP uses VLM-predicted coarse physical motion priors to guide video generation. CausalMotion additionally represents critical physical states as explicit keyframe images, aligns them with the predicted trajectories, and combines appearance references from neighboring keyframes and the preceding latent frame to guide transitions between states.
  • Compared with PhyRPR, CoECT, and DeforM: These methods explore physical reasoning and motion planning, event-level causal representations, and localized spatiotemporal guidance, respectively. CausalMotion focuses on the explicit correspondence between keyframe states and object-centric trajectories and their joint integration through training-free latent guidance.

Additional quantitative comparisons. To provide a more comprehensive comparison, we have added VBench and VLM-as-judge evaluations for VLIPP and consolidated the available results for LTX-Video, VLIPP, VChain, and CausalMotion in Table A.

Table A. Quantitative comparison with closely related methods. PhyGenBench evaluates physical plausibility, VBench reports perceptual quality (%), and VLM-as-judge evaluates physical plausibility (Phys.), temporal consistency (Temp.), semantic alignment (Sem.), and overall quality.

MethodPhyGenBench ↑VBench ↑Phys. ↑Temp. ↑Sem. ↑Overall ↑
LTX-Video0.39080.572.533.891.682.70
VLIPP0.60076.833.503.754.003.75
VChain—78.49————
CausalMotion0.6682.524.054.654.404.37

The VLIPP VBench and VLM-as-judge results are from our additional evaluations. The PhyGenBench results are reported in the revised paper's Table 1, while the VBench comparison uses the same 20 evaluation prompts as VChain, as described in Sec. 4.2.

Regarding VChain reproducibility, we examined its public project and repository and attempted to contact the authors for the complete implementation. However, the code required to reproduce its video generation and evaluation pipeline remains unavailable to us. Since VChain involves sparse inference-time LoRA tuning, an independent reimplementation without the complete configurations could introduce substantial implementation differences.

AuthorsOpenReview ↗

Response to Reviewer NbMM (2/7)

2/

W2: The gains are small and have no error bars. The main-table improvement over VLIPP is 0.05, and the ablations move the average by 0.01 (0.654 → 0.642 / 0.645). With a GPT-Image-1.5 keyframe generator and a 72B VLM in the loop, this is well within the noise one would expect from sampling variance. No seeds, no confidence intervals, no number of prompts per category.

Q2: Please run the full model and the ablations with at least 3 seeds and report mean ± std. Are the 0.01 ablation differences significant?

We thank the reviewer for highlighting the importance of statistical reliability. To address this concern, we conduct multi-seed evaluations and report standard deviations and 95% confidence intervals to provide a more comprehensive assessment of our results.

We evaluated CausalMotion and its ablations using three sampling seeds (42, 123, and 2026) and report mean ± standard deviation and confidence interval. The full model achieves higher mean scores across PhyGenBench, VBench, and VLM-as-judge evaluations. All variants use the same evaluation prompts and video generation configuration. For VBench, we use the same 20 prompts as in the original comparison. For VLM-as-judge, each generated video is evaluated five times, and the scores are averaged to reduce evaluation variability. These repeated judgments are distinct from the three video-generation seeds.

Table B. Multi-seed PhyGenBench evaluation of CausalMotion and its ablations. Results are reported as mean ± standard deviation across three sampling seeds. Δ denotes the difference between the full model's mean score and each ablation's mean score.

del VariantMean ± Std ↑95% CIΔ vs. Full
w/o keyframe reasoning0.471 ± 0.021[0.419, 0.523]+0.191
Keyframe-only (uniform placement)0.642 ± 0.010[0.617, 0.667]+0.020
w/o latent trajectory guidance0.645 ± 0.006[0.630, 0.660]+0.017
w/ simplified guidance0.649 ± 0.006[0.634, 0.664]+0.013
CausalMotion (Full)0.662 ± 0.010[0.637, 0.687]—

As shown in Table B, CausalMotion achieves the highest mean PhyGenBench score of 0.662 ± 0.010, with a 95% CI of [0.637, 0.687]. These results show that the full framework retains higher mean physical plausibility scores across the three sampling seeds. The additional trajectory-related components provide improvements beyond keyframe reasoning alone, although their smaller performance differences require further statistical interpretation.

Table C. Multi-seed VBench Quality and VLM-as-judge Overall scores.

Model VariantVBench Mean ± Std ↑95% CIVLM Overall Mean ± Std ↑95% CI
w/o keyframe reasoning80.91 ± 0.05[80.79, 81.03]3.20 ± 0.21[2.68, 3.72]
Keyframe-only (uniform placement)82.44 ± 0.07[82.27, 82.61]3.37 ± 0.06[3.22, 3.52]
w/o latent trajectory guidance82.46 ± 0.06[82.31, 82.61]3.55 ± 0.11[3.28, 3.82]
w/ simplified guidance81.62 ± 0.04[81.52, 81.72]3.70 ± 0.08[3.50, 3.90]
CausalMotion (Full)82.52 ± 0.03[82.45, 82.59]4.37 ± 0.05[4.25, 4.49]

The full method also achieves the highest mean VBench and VLM-as-judge scores among the ablated variants. Notably, the 95% CI of CausalMotion's VLM Overall score does not overlap with those of the ablation variants, providing additional evidence of a consistent advantage in the evaluated high-level video properties.

The multi-seed results are reported in revised Tables 2–3, with the sampling configuration described in Appendix D.3 and the repeated VLM evaluation protocol in Appendix E.

AuthorsOpenReview ↗

Response to Reviewer NbMM (3/7)

W3: Most of the gain seems to come from keyframes, not the proposed trajectory machinery. Removing keyframe reasoning drops the score to 0.471, while removing trajectory planning or latent guidance barely changes it. The paper's novel parts (Sections 3.2–3.4) are therefore weakly supported. A stronger baseline is needed: LTX-Video conditioned on the same GPT-Image-1.5 keyframes at uniformly spaced indices, with no trajectory at all. The "w/o trajectory physical state" row may already be this, but it is not described precisely enough to tell.

Q3: What exactly does "w/o trajectory physical state" do? Is it LTX-Video + the same keyframes at uniform indices? If not, please add that baseline.

We appreciate the reviewer’s suggestion to further isolate the contribution of trajectory guidance from keyframe conditioning. We clarify the existing keyframe-only baseline and provide additional controlled ablations to examine the contributions of different components.

The requested LTX-Video + identical keyframes + uniform temporal placement baseline is already included in our ablation study. We have renamed it Keyframe-only (uniform placement) in Sec. 4.3, Table 3 in revised paper, to make its configuration explicit.

Keyframe-only baseline. This variant uses exactly the same GPT-Image-1.5-generated keyframes as full CausalMotion, but disables: VLM-based physical trajectory planning (Sec. 3.2), Trajectory-based temporal alignment (Sec. 3.3), Trajectory-guided latent updates (Sec. 3.4). Instead, we use LTX-Video's native multi-keyframe conditioning mechanism and distribute the keyframes uniformly along the video timeline.

For a video containing (T) frames and (K) keyframes, their timestamps are defined as

tj=round⁡(j(T−1)K−1),j=0,…,K−1.t_j=\operatorname{round}\left(\frac{j(T-1)}{K-1}\right), \qquad j=0,\ldots,K-1.

The first and last keyframes are fixed at the video boundaries, while intermediate keyframes are placed at uniform temporal intervals.

As shown in Table B, this baseline achieves 0.642 ± 0.010 on PhyGenBench, compared with 0.662 ± 0.010 for full CausalMotion. This comparison controls for the generated keyframes and shows that trajectory-related mechanisms provide additional mean performance gains beyond keyframe conditioning alone.

Additional ablation: linear 2D trajectory guidance. To further isolate the effect of continuous 2D motion constraints, we introduce a Keyframes + Linear 2D Trajectory Guidance variant. This variant uses the same keyframes and uniformly spaced timestamps as the keyframe-only baseline, but does not use VLM-predicted physical trajectories. Instead, we extract corresponding object bounding boxes from adjacent keyframes and linearly interpolate them to obtain intermediate trajectories:

b(t)=(1−β)bj+βbj+1,β=t−tjtj+1−tj.b(t)=(1-\beta)b_j+\beta b_{j+1}, \qquad \beta=\frac{t-t_j}{t_{j+1}-t_j}.

We then apply the same latent-space guidance mechanism described in Sec. 3.4 using these interpolated trajectories.

This simplified guidance variant achieves a mean PhyGenBench score of 0.649, compared with 0.642 for keyframe-only conditioning. Because these variants use the same keyframes and temporal indices, the comparison more directly isolates the effect of continuous 2D trajectory constraints between keyframes.

Taken together, these comparisons support the complementary roles of our components: keyframe reasoning provides discrete causal states, while trajectory planning, temporal alignment, and latent-space guidance further constrain their timing and continuous evolution. The configurations and results are clarified in Sec. 4.3 (Table 3).

Reviewer_Em26OpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This article proposes CausalMotion, a training free physics guided video generation framework. This method utilizes visual language models to generate keyframes and object motion trajectories, and integrates structured state information into pre trained video diffusion models through time alignment and latent space guidance. The experimental results show that this method achieves good performance on PhyGenBench and VBench, and can improve the physical rationality and temporal consistency of generated videos.

Critical strengths

The paper addresses the issues of physical discontinuity and unreasonable motion in video generation, decoupling state inference from video generation. The overall framework is clear and has certain research value.

By combining keyframe inference, object trajectory planning, and latent space guidance, video generation can be improved without additional training and has good flexibility.

The method achieved good results on PhyGenBench and VBench, and demonstrated the roles of each module through ablation experiments and qualitative visualization.

Critical weaknesses

The ablation experiment shows that the main performance improvement comes from keyframe inference, while the independent gains of physical trajectory planning and latent space guidance are relatively small. Lack of sufficient comparison with simple trajectory methods under the same keyframe conditions.

The method mainly relies on VLM to predict the state of two-dimensional objects and perform trajectory interpolation, lacking quantitative verification of physical constraints such as collisions and forces, making it difficult to fully support the relevant claims of physical causal reasoning. The appendix shows that the IoU of keyframes and predicted trajectories decreases over time, indicating the presence of cumulative alignment errors. In addition, some latent space guidance parameters and implementation details are not clear enough, which affects the reproducibility of the method.

The method relies heavily on large VLMs and image generation models, lacking comparisons under a unified computational budget. Meanwhile, the experimental video is only about 4 seconds long and lacks longer videos and statistical analysis generated multiple times, making it difficult to fully support the conclusion of long-term consistency.

Questions for authors

The ablation gain of trajectory planning and latent space guidance is relatively small. Can we supplement the comparison with simple trajectory methods?

Can you provide quantitative verification of physical constraints such as collisions and gravity?

How to alleviate the accumulation of alignment errors over time? How to determine the key guiding parameters?

Can you provide experimental results for longer videos and complex interactive scenes?

The method takes about 448 seconds for a single generation. Can you provide a performance comparison under a unified calculation budget?

7 comments
AuthorsOpenReview ↗

Response to Reviewer Em26 (1/5)

We sincerely thank the reviewer for the thoughtful feedback and constructive suggestions. We appreciate the opportunity to further clarify the contributions of CausalMotion. We have strengthened the revised manuscript with additional ablation studies, implementation details, long-horizon experiments, and computational efficiency analysis. We address each concern below.


W1:The ablation experiment shows that the main performance improvement comes from keyframe inference, while the independent gains of physical trajectory planning and latent space guidance are relatively small. Lack of sufficient comparison with simple trajectory methods under the same keyframe conditions.

Q1:The ablation gain of trajectory planning and latent space guidance is relatively small. Can we supplement the comparison with simple trajectory methods?

Thank you for suggesting a controlled comparison to better isolate the contribution of trajectory planning. We have clarified the original keyframe-only baseline and added a simplified 2D trajectory guidance baseline using the same generated keyframes.

  1. Keyframe-only baseline.

    Our original ablation study (Sec. 4.3, Table 3) includes a keyframe-only (uniform placement) baseline. This setting uses the same generated keyframes as the full CausalMotion model but disables VLM-based trajectory planning (Sec. 3.2), trajectory-based temporal alignment (Sec. 3.3), and latent trajectory guidance (Sec. 3.4). Instead, it relies on LTX-Video's native multi-keyframe conditioning mechanism, with keyframes uniformly placed at:

    tj=round⁡(j(T−1)K−1),j=0,…,K−1,t_j=\operatorname{round}\left(\frac{j(T-1)}{K-1}\right),\quad j=0,\ldots,K-1,

    where TT is the total number of video frames and KK is the number of generated keyframes. The first and last keyframes are fixed at the beginning and end of the video, respectively.

    This baseline achieves an overall PhyGenBench score of 0.642, compared with 0.662 ± 0.010 for the full CausalMotion model.

  2. Additional ablation: Simplified 2D trajectory guidance.

    To further isolate the contribution of continuous trajectory guidance, we introduce a w/ simplified guidance baseline. It uses the same generated keyframes and uniform temporal indices as the keyframe-only baseline, but additionally applies latent-space guidance using linearly interpolated 2D trajectories, without VLM-based physical trajectory prediction.

    Specifically, we extract the bounding boxes of corresponding objects from adjacent keyframes and interpolate them as:

    b(t)=(1−β)bj+βbj+1,β=t−tjtj+1−tj,b(t)=(1-\beta)b_j+\beta b_{j+1}, \qquad \beta=\frac{t-t_j}{t_{j+1}-t_j},

    where bjb_j and bj+1b_{j+1} denote the bounding boxes of the same object in adjacent keyframes. The interpolated trajectories are then incorporated using the same latent-space guidance mechanism as the full model. The results are summarized below.

Table A. PhyGenBench ablation results comparing keyframe-only conditioning, simplified 2D trajectory guidance, and the full CausalMotion framework.

MethodMechanics ↑Overall ↑
Keyframe-only (uniform placement)0.5830.642 ± 0.010
w/o latent trajectory guidance0.5940.645 ± 0.006
w/ simplified guidance0.5830.649 ± 0.006
CausalMotion (Full)0.6080.662 ± 0.010

Under identical generated keyframes and uniform temporal placement, introducing linear 2D trajectory guidance improves the overall score from 0.642 to 0.649, demonstrating that continuous motion constraints provide additional benefits beyond discrete keyframe conditioning. These complementary evaluations indicate that trajectory guidance contributes beyond keyframe reasoning, with benefits that vary across physical phenomena.

We have also clarified the ablation settings and incorporated the additional comparison in Sec. 4.3, Table 3 of the revised manuscript.

AuthorsOpenReview ↗

Response to Reviewer Em26 (2/5)

W2: The method mainly relies on VLM to predict the state of two-dimensional objects and perform trajectory interpolation, lacking quantitative verification of physical constraints such as collisions and forces, making it difficult to fully support the relevant claims of physical causal reasoning.

Q2: Can you provide quantitative verification of physical constraints such as collisions and gravity?

We thank the reviewer for suggesting a more direct quantitative evaluation of physical constraints. We conducted an exploratory kinematic analysis of generated videos, focusing on gravity-driven motion and collision-related bouncing behavior.

  1. Physical Kinematic Evaluation.

    Motivated by the constant-acceleration constraints in NewtonRewards [1], the free-fall analysis in PISA [2], and studies of collision and restitution behavior [3, 4], we introduce two tracking-based metrics.

    Gravity-kinematic Consistency. For detected downward-motion segments, we fit the tracked vertical object positions using a quadratic motion model: y^(t)=c0+c1t+12at2.\hat y(t)=c_0+c_1t+\frac{1}{2}at^2. We measure consistency with constant-acceleration motion using the normalized root mean squared fitting error: Eg=1T∑t(y(t)−y^(t))2d,E_g= \frac{ \sqrt{\frac{1}{T}\sum_t(y(t)-\hat y(t))^2} }{d}, where y(t)y(t) is the tracked vertical position, dd is the median object diameter within the segment, and TT is the number of valid observations.

    We define gravity-kinematic consistency as: Cg=1[Eg≤0.25;∧;aΔt2d>0.05],C_g=\mathbb{1}\left[ E_g\leq0.25 ;\land; \frac{a\Delta t^2}{d}>0.05 \right],

    where Δt\Delta t denotes the observed segment duration.

    Approximately Non-amplifying Motion Reversal. For elastic-impact scenarios, we examine the relationship between vertical velocities before and after a downward-to-upward motion reversal, motivated by the restitution principle discussed in [3] and the physics-based bouncing evaluation in Principia [4].

    Specifically, we estimate the vertical velocities by linearly fitting six tracked frames on each side of the detected reversal and compute: rv=∣vafter∣∣vbefore∣.r_v=\frac{|v_{\mathrm{after}}|}{|v_{\mathrm{before}}|}. We then define R: downward-to-upward reversal detected,

    \text{R} \;\land\; (0<r_v\leq1.05) \right].$$ The upper bound of 1.05 allows a 5% measurement tolerance. Both metrics use heuristic normalization and thresholds. The second metric serves as a bounce-related kinematic proxy, rather than a direct measurement of collision restitution or energy conservation.
  2. Quantitative Results.

    We evaluated 10 physics-related prompts with three random seeds each, comparing full CausalMotion against the keyframe-only baseline using object detection and trajectory tracking. Across the complete 10-prompt evaluation, both methods achieved a gravity-kinematic consistency rate of 46.67%. We further examined a post-hoc subset of five elastic-ball impact prompts

Table B. Exploratory kinematic evaluation on five elastic-ball impact prompts with three paired random seeds (15 videos per method). Higher values indicate greater consistency with the specified kinematic criteria.

MetricCausalMotion (Full)Keyframe-only
Gravity-kinematic Consistency ↑46.67% (7/15)20.00% (3/15)
Approximately Non-amplifying Motion Reversal ↑86.67% (13/15)60.00% (9/15)

Within this subset, full CausalMotion outperforms keyframe-only conditioning by 26.67 percentage points on both metrics. These results suggest that combining VLM-based physical state planning with trajectory guidance can improve observable motion consistency in scenarios involving gravitational descent and bouncing. These exploratory results support improved kinematic consistency in this subset, although the gains do not extend uniformly to the complete evaluation.

We further clarify that physical reasoning in CausalMotion refers to using VLM-based physical commonsense to infer object state transitions and motion trends, rather than explicitly solving force equations or performing physical simulation.

[1] "What about gravity in video generation? Post-Training Newton's Laws with Verifiable Rewards." arXiv preprint arXiv:2512.00425 (2025).

[2] PISA experiments: Exploring physics post-training for video diffusion models by watching stuff drop[C]//Proceedings of the 42nd International Conference on Machine Learning. PMLR, 2025, 267: 35685–35709.

[3] "The psychophysics of bouncing: Perceptual constraints, physical constraints, animacy, and phenomenal causality." PloS one 18.8 (2023): e0285448.

[4] "Principia: Relational physics tests for video models." arXiv preprint arXiv:2609.04200 (2026).

AuthorsOpenReview ↗

Response to Reviewer Em26 (3/5)

W3.1: The appendix shows that the IoU of keyframes and predicted trajectories decreases over time, indicating the presence of cumulative alignment errors.

Q3.1: How to alleviate the accumulation of alignment errors over time?

Thank you for highlighting the temporal alignment behavior reported in Appendix A. The decreasing IoU in Figs. 5–6 measures the spatial agreement between independently generated keyframe object states and VLM-predicted trajectories, rather than the motion error of the final synthesized video. Because keyframe generation and trajectory planning operate independently, their object locations and temporal parameterizations may differ. CausalMotion addresses this discrepancy through two complementary mechanisms.

  1. Trajectory-based temporal alignment.

    As described in Sec. 3.3 (Eq. 4), we assign each generated keyframe to the trajectory timestamp that maximizes the IoU between their corresponding object bounding boxes, subject to monotonic temporal ordering.

    This establishes temporal correspondence between the two representations and reduces discrepancies caused by their different temporal parameterizations, without directly modifying the planned trajectories.

  2. Soft latent-space guidance.

    As described in Sec. 3.4 (Eqs. 5–6), we use expanded target bounding boxes and Gaussian soft masks to perform localized, weighted latent updates rather than enforcing trajectories as hard spatial constraints. The reference features combine appearance anchors from neighboring keyframes with warped latent features from the preceding temporal position. This allows generation to leverage both keyframe appearance information and predicted motion, rather than relying exclusively on a single trajectory estimate.

    Together, these mechanisms improve tolerance to moderate keyframe–trajectory mismatches and reduce their influence on the final generation. We explicitly analyze these behaviors in Appendix A (Figs. 5–6), while the corresponding alignment and guidance mechanisms are detailed in Secs. 3.3–3.4.


W3.2: In addition, some latent space guidance parameters and implementation details are not clear enough, which affects the reproducibility of the method.

Q3.2: How to determine the key guiding parameters?

Thank you for requesting further clarification of the guidance parameters. All experiments use a fixed heuristic configuration shared across prompts, without per-prompt hyperparameter tuning.

  1. Trajectory guidance strength and denoising schedule.

    We adopt a piecewise guidance schedule based on the normalized denoising progress within each sampling stage. Let ss denote the zero-based denoising step, NN the number of steps in the current stage, and ps=s/(N−1)p_s=s/(N-1). The guidance strength is defined as:

\begin{cases} 0, & p_s < 0.2, \\\\ 0.3, & 0.2 \leq p_s < 0.75, \\\\ 0.105, & p_s \geq 0.75. \end{cases}$$ Guidance is applied every two steps, giving the effective strength: $\alpha_s=\mathbb{1}[s\bmod 2=0]\cdot g(p_s).$ We use the distilled LTX-Video 13B two-stage sampling schedule with 7+3 denoising steps, resetting the progress counter for each stage. Consequently, guidance is applied at steps 3, 5, and 7 of the first stage and step 3 of the second stage (one-based indexing), with strengths 0.300, 0.300, 0.105, and 0.105, respectively. 2. Spatial guidance and keyframe conditioning. We use a fixed target bounding-box expansion factor of $\rho=1.1$ and a source crop scale factor of $0.8$. Spatial guidance is implemented using a 2D Gaussian soft mask with standard deviation $\sigma=0.55$ in normalized local bounding-box coordinates. The keyframe conditioning strength is fixed at $\lambda=0.9$ for all keyframes, independently of the trajectory guidance strength $\alpha_s$. Appearance blending weights for neighboring keyframe anchors and the warped preceding latent are determined by their relative temporal positions, rather than separately optimized hyperparameters. 3. Parameter selection rationale. These parameters are selected to balance the planned motion constraints with the pretrained video generator's prior. The guidance schedule avoids additional trajectory updates during early denoising to preserve global structure, applies stronger constraints during intermediate steps to encourage the intended motion, and reduces guidance strength during later steps to limit potential visual artifacts. Similarly, the expanded target regions and Gaussian soft masks provide tolerance to moderate trajectory prediction errors. The configuration is kept identical across prompts rather than optimized individually for each video. It is a fixed heuristic configuration, not the result of an exhaustive hyperparameter search. We have provided the detailed implementation settings in Secs. 3.4–3.5 and Appendix D.2–D.3 (Algorithm 2, Eqs. 7–8) to facilitate reproducibility.
Reviewer_rooqOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

CausalMotion is a paper that presents a way to improve physical consistency in video generation by combining VLM reasoning, trajectory guidance, and generated image references for keyframes. The main selling point is that no additional training is required. The base model for video generation is LTX-Video, and the authors use Qwen2.5-VL for physical reasoning and GPT-Image-1.5 for keyframe generation. First, a VLM is used to reason about how a physical event evolves over time, and a sequence of keyframes is generated to represent intermediate states. The VLM is also used to predict object positions and motion trajectories using 2D bounding boxes and velocities, which are then interpolated over time. The keyframes are aligned with the trajectories, which are subsequently used to guide the latent space of the video diffusion model during generation. The paper reports improvements over several SOTA methods on physical consistency benchmarks, and qualitative results are shown to demonstrate more physically consistent transformations.

Critical strengths

This paper addresses an important problem of improving physical consistency in video generation, which remains a known limitation of many existing video generation methods. The proposed approach does not require additional training or fine-tuning, and uses generated intermediate keyframes and trajectory guidance to improve physical transformations, making it relatively easy to adopt. The keyframes can potentially be used for visual inspection and adjusted separately. The qualitative results demonstrate that the method works better than several existing methods on a number of examples, and some of the results look quite plausible. The proposed approach could potentially be applied to other video generation models whenever they produce physically inconsistent results.

Critical weaknesses

My main concern is that the proposed techniques are not new, and the overall system seems quite incremental. It combines existing VLM reasoning models, keyframe generation techniques, and latent-space guidance methods. It seems that the main improvements come from the generated keyframes rather than the physical trajectory reasoning, as the ablation studies indicate that removing keyframe reasoning significantly reduces performance. The use of 2D bounding boxes and velocities as guidance constraints also seems quite limiting, as it does not explicitly account for more complex 3D trajectories or physical interactions. A more detailed limitations study would be helpful. The restriction to 4-second videos is also rather limiting, especially given the paper's focus on long-horizon physical consistency.

Questions for authors

1/ Have you compared standard LTX-Video conditioning using the same generated keyframes without additional trajectory guidance? How important is the 2D trajectory guidance qualitatively, and what are its main limitations?

2/ Given the focus on long-horizon physical consistency, why are the experiments limited to 4-second videos? What would happen with longer sequences?

6 comments
AuthorsOpenReview ↗

Response to Reviewer rooq (1/5)

We sincerely thank the reviewer for the thoughtful feedback and constructive suggestions. Please find below our responses to the comments raised by the Reviewer.


W1: My main concern is that the proposed techniques are not new, and the overall system seems quite incremental.

We thank the reviewer for the thoughtful comments on the novelty of our approach.

The contribution of CausalMotion lies in jointly integrating sparse causal states and continuous object-centric motion constraints into a pretrained video diffusion model without parameter updates.

Specifically, our framework introduces two closely related mechanisms:

  1. Trajectory-based keyframe temporal alignment (Sec. 3.3). Instead of uniformly distributing keyframes over time, we determine their temporal positions by matching object states in the generated keyframes with predicted motion trajectories. This aligns critical physical states with the planned motion dynamics.
  2. Localized latent-space guidance with keyframe appearance references (Sec. 3.4). We construct spatiotemporal guidance regions from object trajectories. During diffusion sampling, we perform localized latent updates using appearance references from neighboring keyframes and warped preceding latent features, encouraging continuous motion between key states.

Compared with keyframe-reasoning approaches such as VChain, CausalMotion explicitly establishes temporal correspondences between keyframes and object trajectories and provides continuous object-level motion constraints during inference, without requiring test-time LoRA adaptation.

To further clarify these contributions, we provide controlled ablations that distinguish the effects of keyframe conditioning, 2D trajectory guidance, and physics-aware state planning, as detailed below.

AuthorsOpenReview ↗

Response to Reviewer rooq (2/5)

Q1.1: Have you compared standard LTX-Video conditioning using the same generated keyframes without additional trajectory guidance?

We thank the reviewer for this important question.

The requested keyframe-only baseline is included in our ablation study (Sec. 4.3, Table 3), under Keyframe-only (uniform placement). We clarify its implementation and provide an additional controlled experiment to isolate the contribution of 2D trajectory guidance.

  1. Keyframe-only baseline.

    This baseline uses exactly the same generated keyframes as the full CausalMotion model, but disables VLM-based physical trajectory planning (Sec. 3.2), trajectory-based temporal alignment (Sec. 3.3), and latent trajectory guidance (Sec. 3.4). Instead, we directly use LTX-Video's native multi-keyframe conditioning mechanism, placing all keyframes uniformly along the video timeline. For a video with (T) frames and (K) keyframes, their temporal indices are assigned as tj=round⁡(j(T−1)K−1),j=0,…,K−1.t_j=\operatorname{round}\left(\frac{j(T-1)}{K-1}\right),\quad j=0,\ldots,K-1.

    As shown in Table 3 of the revised manuscript, this baseline achieves an average PhyGenBench score of 0.642, compared with 0.662 ± 0.010 for the full CausalMotion model.

Table A: Ablation results on PhyGenBench. Higher scores indicate better physical plausibility.

Model variantAvg. ↑
w/o Keyframe Reasoning0.471
Keyframe-only (Uniform Placement)0.642 ± 0.010
Trajectory-Aligned Keyframes (w/o Latent Guidance)0.645 ± 0.006
Keyframes + Linear 2D Trajectory Guidance0.649 ± 0.006
Full CausalMotion0.662 ± 0.010
  1. Additional controlled ablation: 2D trajectory guidance.

    This setting uses the same generated keyframes and uniformly assigned temporal indices as the keyframe-only baseline, without VLM-based physical trajectory prediction. Instead, we linearly interpolate the bounding boxes of corresponding objects in adjacent keyframes to construct intermediate 2D trajectories:

    \qquad \beta=\frac{t-t_j}{t_{j+1}-t_j}.$$ We then apply these interpolated trajectories using the same latent-space guidance mechanism described in Sec. 3.4. This variant achieves 0.649 on PhyGenBench, improving upon the keyframe-only baseline (0.642). Since both settings use identical keyframes and conditioning timestamps, this comparison directly isolates the additional contribution of trajectory-guided latent updates. These results support the complementary roles of keyframe reasoning and trajectory guidance: keyframes specify causally related physical states, while trajectory planning, temporal alignment, and localized latent guidance further constrain when and how objects transition between these states.
AuthorsOpenReview ↗

Response to Reviewer rooq (3/5)

Q1.2: How important is the 2D trajectory guidance qualitatively?

We appreciate the reviewer's insightful question regarding the qualitative benefits of 2D trajectory guidance. 2D trajectory guidance improves generation in two complementary ways: aligning keyframe timing with object motion and encouraging temporal continuity between keyframes.

  1. Trajectory-based temporal alignment.

    In Sec. 3.3, we use predicted object trajectories to determine the temporal positions of keyframes rather than assigning them at uniform intervals.

    For example, an object falling under gravity changes velocity over time, so its intermediate physical states need not occur at equally spaced timestamps. Uniform keyframe placement constrains the initial, intermediate, and final appearances but does not explicitly account for these motion dynamics. In contrast, trajectory-based alignment matches keyframe states to the predicted object motion, encouraging more physically consistent timing of state transitions.

    We provide a controlled qualitative comparison in Appendix B.2 (Fig. 9). Both settings use identical prompts, generated keyframes, and random seeds, with latent trajectory guidance disabled to isolate the effect of temporal alignment. With uniform placement, the object contacts a pillow but subsequently moves back into the air, producing an inconsistent motion sequence. In contrast, trajectory-based alignment yields a more coherent progression: the object descends, contacts the pillow, and remains at rest.

  2. Improved temporal continuity between keyframes.

    When using only LTX-Video's native multi-keyframe conditioning, the generated keyframes constrain visual states at selected timestamps, but the model must infer the intermediate motion from its learned priors. We observe that this can produce piecewise-looking temporal transitions: individual intervals may appear locally smooth, while motion and appearance evolution across intervals lack global consistency.

    To address this, our latent-space guidance mechanism (Sec. 3.4) constructs frame-wise spatial guidance regions from the predicted 2D trajectories.

    Appendix B.3 (Fig. 10) provides additional consecutive-frame visualizations of the keyframe-only setting without latent trajectory guidance. Across three examples, five consecutive frames around keyframe conditioning timestamps reveal temporary blur and deformation of the main objects.

    Together, these qualitative observations and quantitative results support the role of 2D trajectory guidance in improving both physically meaningful state timing and temporal coherence between keyframes.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Global Response

Dear Reviewers,

We sincerely thank all reviewers for your thoughtful evaluations and constructive feedback. Beyond our individual responses, we would like to highlight the key contributions of CausalMotion and summarize the additional experiments conducted during the rebuttal.

[Our Contributions]

  • Training-free physical reasoning framework that decouples physical state planning from video synthesis.
  • Joint keyframe–trajectory guidance through explicit temporal alignment and localized latent-space updates, without additional training.
  • Improved physical plausibility and video quality, achieving 0.66 on PhyGenBench and 82.52% on VBench.

[New Experiments]

We have conducted additional experiments and updated the manuscript to address the reviewers' concerns:

  • Controlled ablations with matched keyframes and simplified 2D trajectory guidance (rooq, Em26, NbMM, VfBk).
  • Multi-seed evaluations with standard deviations and confidence intervals (Em26, NbMM).
  • Alternative keyframe generators: use FLUX.2, Qwen-Image-2512, and Z-Image-Turbo (Em26, NbMM).
  • Independent VLM evaluations: use GPT-4o and GLM-5.3-Flash (NbMM).
  • Longer video generation: extend to 6-second and 8-second sequences (rooq, Em26).
  • Inference-budget and physical-motion analyses, including Best-of-N comparisons and kinematic evaluations (Em26).
  • Per-prompt paired comparisons to examine the effectiveness of joint keyframe–trajectory guidance (VfBk).

We hope these additional results and clarifications address the reviewers' concerns and further demonstrate the effectiveness of CausalMotion. We sincerely appreciate your time and consideration.

Yours sincerely,

Authors

AuthorsOpenReview ↗

Discussion Summary

Dear Area Chair,

We sincerely thank you and all reviewers for your time and constructive feedback.

In this paper, we propose a training-free framework that improves physical consistency in video generation by coupling reasoned keyframes and object-centric trajectories through temporal alignment and latent-space guidance.

During the rebuttal, we conducted extensive additional experiments and revised the manuscript to address the reviewers' concerns:

  • Reviewer rooq: Controlled trajectory ablations, qualitative analyses, limitations of 2D trajectories, and 6- and 8-second video generation (Sec. 4.3; Appendices B, G, I).
  • Reviewer Em26: Simplified trajectory comparisons, quantitative physical evaluations, longer videos, and computational budget analysis (Sec. 4.3; Appendices C, G). The reviewer confirmed that all concerns were resolved.
  • Reviewer NbMM: Expanded baseline comparisons, multi-seed statistics, alternative keyframe generators, independent VLM judges, and reproducibility details (Tables 2–3; Appendices C–F).
  • Reviewer VfBk: Matched ablations, paired comparisons, and stage-wise failure analysis (Appendix H). The reviewer acknowledged the strengthened experimental evidence. We further provided preliminary Wan2.2 experiments to address the remaining concern about backbone generalizability.

Although reviewers rooq and NbMM have not provided further feedback, we have addressed their specific concerns through the corresponding responses and manuscript revisions.

We hope these additional results and clarifications will be helpful in your final assessment. Please refer to our rebuttal and revised manuscript for further details. If time permits, we would sincerely appreciate your help in encouraging the remaining reviewers to consider our responses.

Best regards,

Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.