acceptodds
Withdrawn from ICLR 2027

CompAdapt: Adaptable Composite Motion Modeling for Physics-Consistent Text-to-Video Generation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

While diffusion-based text-to-video (T2V) models have demonstrated impressive capability in generating realistic and temporally coherent videos, they often fail to respect fundamental physical dynamics. Although recent physics-constrained methods incorporate explicit dynamics priors to improve physical plausibility, they remain limited to simple single-type motions, depend on manually specified parameters, and struggle to generalize to unseen physical laws. In this work, we propose CompAdapt, a physics-consistent T2V framework for adaptable genera tion across complex real-world scenarios. It extends neural dynamics modeling beyond single-type motions to encompass composite physical behaviors, including coupled motions, multi-stage transitions, and multi-object collisions. Furthermore, CompAdapt translates natural language prompts into structured physical semantics, enabling end-to-end specification of motion types, temporal relations, and initial physical parameters. To generalize to novel physical environments, CompAdapt introduces dynamics-aware prior matching, achieving one-shot adaptation without retraining the core dynamics module. In addition, a physics-aware latent feature fu sion module improves visual fidelity under fast and complex motion. Experiments on physics-focused T2V benchmarks demonstrate that CompAdapt improves phys ical consistency over both general T2V models and physics-constrained baselines, while preserving high visual quality and adaptability to unseen dynamics.

Primary area:
generative models
Keywords:
text-to-video generation, physics-consistent generation, composite motion modeling, neural dynamics, one-shot adaptation

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_jcusOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

CompAdapt addresses physics-consistent text-to-video generation involving parallel motions, sequential transitions, and object collisions. It parses text into structured motion sequences and initial physical states, composes pretrained Neural Newtonian Dynamics modules, handles collisions through a dedicated interaction module, and conditions video generation on the resulting physical states. For unfamiliar physical environments, it selects a pretrained dynamics prior using a reference trajectory and performs regularized one-shot adaptation. The central claims are that elementary dynamics modules can be composed into physically consistent complex motions and adapted to unfamiliar dynamics from a single observation.

Critical strengths
  1. The problem is meaningful. Extending physics-guided video generation beyond isolated motion types to concurrent motions, temporal transitions, and collisions addresses a substantial limitation of existing approaches.

  2. The system decomposition is conceptually reasonable. Separating continuous dynamics from collision-induced discontinuities is well motivated. Introducing mass explicitly also addresses an important distinction between single-object kinematics and multi-object interactions.

Critical weaknesses
  1. The primary metric does not establish that the intended physical process is generated correctly. The most consequential concern is the validity of the evaluation. PIS in Eq. (21) measures the relative temporal stability of a selected quantity. It does not directly verify the correct physical parameter, initial condition, or governing equation. For example, when acceleration is the monitored quantity, an incorrect but constant gravitational acceleration can receive a perfect score. Moreover, an identically zero monitored quantity also receives a perfect score under Eq. (21), potentially rewarding degenerate outputs that fail to exhibit the requested motion. Table 9 demonstrates this limitation: for Oscillating size, the ground-truth trajectory receives a GT-PIS of 0.106, whereas the adapted model receives 0.955. Appendix E acknowledges that this case illustrates metric behavior rather than adaptation quality, yet the scenario remains included in the near-domain aggregate. The table also describes GT-PIS as an attainable upper bound, although model scores exceed it in several cases. Consequently, the reported results do not adequately distinguish correct target dynamics from trajectories that merely keep the monitored quantity stable. Target-equation residuals, parameter errors, trajectory errors, and collision conservation errors are needed. The paper should also distinguish measurements on predicted state trajectories from measurements extracted from the final generated videos.

  2. The theoretical exactness claims do not establish the correctness of the implemented trajectory composition. Equation (6) combines independently generated trajectories after subtracting a shared force-free trajectory. Appendices C and H claim that this construction is also exact for affine-linear dynamics acting on shared coordinates. However, solutions of different linear dynamical systems cannot generally be composed by adding their independent rollouts. Consider constant acceleration combined with linear drag: v˙=a−γv,v(0)=0.\dot v=a-\gamma v,\qquad v(0)=0. The acceleration-only module produces �0��0�. The drag-only module, starting from rest, produces �1��1�, and the force-free velocity is also zero. Equation (6) therefore gives vsyn(t)=at,v_{\mathrm{syn}}(t)=at, whereas the correct combined solution is: v(t)=aγ(1−e−γt).v(t)=\frac{a}{\gamma}\left(1-e^{-\gamma t}\right). This counterexample involves entirely linear dynamics. The superposition argument in Eq. (15) requires responses governed by the same system matrix AA, which is not established for the different category-specific pretrained modules used by the method. Recomputing the composition at sufficiently small time steps could yield an approximation, but would require a corresponding numerical analysis rather than a general exactness claim. A related gap concerns collisions. Equations (17)–(19) establish conservation for an analytical impulse map, while the main method describes a learned collision module. Showing that a network can represent a conserving map does not establish that the trained network necessarily conserves those quantities. The claim of “guaranteed physical consistency” therefore exceeds the demonstrated guarantees.

  3. The reported comparisons do not establish the benefit of one-shot adaptation In Table 9, the adapted Var-PIS is lower than the prior-matching-only ZS-PIS in 32 of 34 scenarios. Examples include: Standard deceleration: 0.997 → 0.574, Combined lunar gravity, low friction, and size change: 0.998 → 0.904, Tight spiral: 0.700 → 0.459. Given the metric limitations discussed above, these decreases do not necessarily imply worse physical accuracy. Nevertheless, they show that the paper’s own primary metric does not establish the benefit of adaptation. The main discussion emphasizes the absolute post-adaptation scores without adequately explaining this systematic decrease. Furthermore, 22 of the 34 controlled scenarios involve parameter shifts, while the average score for structurally unfamiliar far-domain scenarios is only 0.515. The evidence is therefore stronger for adjustment within existing dynamics families than for broad adaptation to unfamiliar physical laws. The three real-video experiments also leave the independent testing and calibration protocols insufficiently specified. It remains unclear whether adaptation identifies transferable environmental dynamics or primarily fits the observed motion.

  4. The experimental controls do not isolate the contributions of the proposed components The ablations in Table 2 replace all parallel motions with sequential execution or all sequential motions with synchronous execution. These comparisons demonstrate that respecting the requested temporal relationships matters, but do not establish that the proposed composition rule is superior to other reasonable composition methods. A more informative comparison would hold the parser and video generator fixed while replacing the proposed dynamics composition with joint integration, an analytical physics solver, or simple parameter identification. The video-generation comparison is also confounded: the proposed generator is fine-tuned on task-specific data, whereas Table 3 compares it against untuned Wan-Move. The reported gains therefore cannot be attributed specifically to spatial warping and adaptive blending. Additionally, describing Wan-Move’s feature propagation as supporting only translation is too restrictive. The original method uses dense point trajectories and demonstrates rotation control; different point trajectories can collectively encode rotation. A comparison using equivalent motion information and matched training data is necessary to establish the added value of the proposed rendering components.

Questions for authors

Refer to Critical Weaknesses

Reviewer_5nEuOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes a " physics-consistent text-to-video (T2V)" method that improve physics informed video generation beyond single motion regimes to multi dynamics with collisions and one-shot adaptation to other physical laws.

The method mixes language into structured physical semantics like motion types, temporal composition, and initial parameters. then, it composes per type neural ODE via “free-plus-particular” scheme, models discrete collision updates with a separate neural collision module and renders videos using a physics-aware latent warping and blending mechanism on top.

A prior-matching strategy selects the best pre-trained dynamics prior for fast one-shot adaptation, regularized to retain physical priors. Experiments on curated physics-focused datasets report higher physical plausibility (PIS) than general T2V and physics-constrained baselines while maintaining comparable visual fidelity, and show promising one-shot adaptation on out-of-distribution (OOD) dynamics.

Critical strengths
  • I think the problem is very relevant and the presentation of the method good.

  • they extends prior physics-informed T2V (e.g., NewtonGen) from single-type motions to composite dynamics with an explicit rule (free-plus-particular superposition) and temporal chaining.

  • Introduces a neural discrete collision module that maps pre to post collision, explicitly handling discontinuities that continuous ODE-based models struggle with.

  • proposes a dual-LLM text to physical that separate motion semantics (types and temporal relations) from parameter estimation, bridging free-form language to structured dynamics inputs.

  • I like the ablation studies effort where they show effects of the single components.

Critical weaknesses
  • I see many internally constructed datasets and the PIS metric inherited from NewtonGen; limited use of external, standardized evaluation (e.g., WorldBench, PhysicsIQ), making comparability harder.

  • how exactly are the closed-sourceT2V systems (e.g., Sora, Veo3) used in not clear.

  • The collision benchmarks seems limited to two rigid objects and elastic collisions.

  • the metric (PIS, the Physical Invariance Score) rewards any constant velocity or acceleration, including the wrong one of one quantity along one trajectory (Eq. 21). what is needed in my opinion is a trajectory error against ground truth, or the recovered parameter.

  • wan move generates video from a first frame, so where does that frame come from? If it’s the ground-truth first frame, this is image-to-video, not text-to-video

  • how do units in the prompt (e.g. “0.9 meters”, Table 13) become image coordinates?

  • Is “Ours” PIS measured on the planned trajectory Z(t) or re-extracted from the generated video the way baselines must be? This is important of a fair comparison.

  • The state, modules, pretrained weights, metric, data protocol and baselines all come from NewtonGen. For now, I have the impression that the new parts is a mixed of standard techniques: an LLM parser, superposition of motions, textbook impulse collisions, picking the pretrained model that fits best, fine-tuning with a penalty for drifting from pretrained weights (“L2-SP” regularizer), and patch warping. Turning text into planned trajectories with an LLM and then generating video from them is prior work that isn’t discussed in the text. for example.GPT4Motion, LVD, PhysCtrl, PhysGen is cited but not compared against.

  • One simple baseline is missing: the 12 motion types are textbook closed form motions, so a baseline is parser → analytic kinematics or a 2D physics engine → Wan-Move. Why use neural ODE modules? for adaptation, the baseline could be a least squares fit of g/ friction or damping on the single clip. both re very cheap.

  • table 9: “Standard” and “strong thrust” have identical rows. The chaotic double pendulum has std 0.000 across 5 different reference clips, why?

  • All benchmarks are the authors’ own. There are four self-built datasets, no established physics benchmark (they even cite T2VPhysBench but don’t use it), no user study, no seeds or confidence intervals, and no stated number of videos per method.

finally, a little doubt, when comparing (with the help of a LLM) Table 1 with Table 1 of NewtonGen (Yuan et al., ICLR 2026), which reports PIS for single motions, I found that many entries for the composite-motion scenarios are identical to four decimals to entries in that table. For example:

the v_y row of “Approaching + Rotation → Post-collision Rebound” (Sora 0.6510, Veo3 0.8384, CogVideoX 0.6690, Wan2.2 0.8481, PhysT2V 0.8913, Ours 0.9371, Reference 0.9986) contains the same values as NewtonGen’s “3D Motion” v_y row, but several baseline values are assigned to different methods (0.8481 is Sora in NewtonGen and Wan2.2 here; 0.8913 is Veo3 there and PhysT2V here). The Reference column and most “Ours” entries also match NewtonGen’s reported values. In addition, some rows are identical across different composite scenarios (e.g., the v rows of “Circular Motion → Post-collision Linear”, “Uniform Acceleration + Scaling → Uniform Motion” and “Damped Oscillation + Rotation → Uniform Translation”). There may be an explanation and this can be explained very quickly but I think is worth to ask.

Questions for authors
  • Where do the Table 1 values come from? Provide per-video PIS and the generated videos for every method on the composite prompts. can you release the evaluation script? Since Table 1 is the main evidence for the central claim, my assessment depends on this doubt being resolved.

  • Report trajectory error and recovered physical parameters (e.g. estimated g) for adaptation and for Table 1.

  • Where does the first frame come from, how are units converted to image coordinates, and how are stage transition times set?

  • Is PIS for “Ours” measured on the generated video, using the same extraction as the baselines?

  • Id like to see the baseline and the parameter-fitting baseline for adaptation described in Weaknesses.

Reviewer_ChuUOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

CompAdapt is a physically consistent text to video generation framework.

  • The paper builds up on prior work (NewtonGen) with a Neural Newtonian Dynamics approach, and extends this to include an additional 10th state for mass.
  • The authors include two fine tuned Qwen Loras for text to physics parsing and compose motion into parallel, sequential and collision using trained models for each.
  • The paper designs a one-shot dynamics adaptation strategy for unseen physical laws.

Experiments include text to video generation with self built datasets and metrics for physical accuracy scored across different models (baselines and prior work) to show the benefits of the proposed approach

Critical strengths
  • Physically controllable T2V generation is an important problem, and a general approach that accounts for multiple object interactions without manually specifying physical parameteres is useful and scalable
  • The approach using language to physics to generation makes it interpretable, and the temporal hierarchy idea makes it easier to split up across time
Critical weaknesses
  1. The metrics don't seem to match and would need some clarifications (a) The numbers in table 1 of the submission has a lot of overlap with the numbers of table 1 from NewtonGen (the approach the paper builds off of) and the numbers quoted by the authors match that of NewtonGen upto 4 decimal places on many values. This would be practically impossible unless the exact same videos and scoring and seed etc are used in both cases since NewtonGen adds a CI of around 0.01 for most values. (b) NewtonGen is open source (the model and evals etc), but the authors don't report the numbers from either the paper or a reproduction and instead report a retrained version of NewtonGen with lower metrics.
  2. The code and models for the proposed approach have not been open sourced, and so this is not possible to reproduce or verify the effects of each of the changes mentioned. All the evaluations are on self built benchmarks, but the numbers for most models match exactly to table 1 on NewtonGen, so this is unclear too. Numbers across public benchmarks would make it easier to understand the significance of the approach.
  3. Novelty over NewtonGen is only incremental and some related work hasn't been cited - MoReGen, PhysGen and other LLM planned physics motion work should be compared and contributions highlighted.
Questions for authors

Evals:

  1. Table 1 has many metrics exactly matching (upto 4 decimal places) the table 1 numbers from NewtonGen labelled as ours, please clarify how these were obtained.
  2. How do the metrics generated in table 1 compare with the metrics from table 1 of NewtonGen? There are many other numbers that match exactly too, so is the dataset used for the generations here a superset of the ones in NewtonGen?
  3. The reproducibility of these metrics and evals can not be confirmed yet, so would need to be addressed. Open sourcing all the code and models would be an option to confirm these. What were the exact datasets used to produce the tables 2 and 3 too?
  4. CI for the metrics would make it easier to understand the significance of the changes and variance.

NewtonGen:

  1. Why are the numbers from NewtonGen not the same as those published in the paper, and why did the authors need to retrain NewtonGen again? Which components of NewtonGen were retrained as part of the effort?

Dynamics adaptation:

  1. Please explain the dynamics adaptation to novel scenarios in more detail. The appendix lists that training is for 100 epochs, but this doesn't count as a one-shot adaptation IIUC.

I would be willing to change my rating if strong evidence is presented towards the table 1 numbers and reproducibility.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.