acceptodds
Under review as a conference paper at ICLR 2027

What about gravity in video generation? Post-Training Newton's Laws with Verifiable Rewards

Abstract

Video generators synthesize visually compelling clips, yet often violate basic physical laws–objects float or collide inconsistently, accelerations drift–revealing a gap between visual and physical realism. We propose NewtonRewards, the first physics-grounded post-training framework for video generation based on verifiable rewards. Instead of relying on human or VLM feedback, NewtonRewards extracts measurable proxies from generated videos using frozen utility models: optical flow serves as a proxy for velocity, while high-level visual features serve as a proxy for physical properties. These proxies enable explicit enforcement of Newtonian structure through two complementary rewards: a Newtonian kinematic constraint enforcing constant acceleration dynamics, and a physical properties conservation reward preventing trivial, degenerate solutions. We evaluate NewtonRewards on five Newtonian Motion Primitives (free fall, horizontal/parabolic throw, and ramp sliding down/up) using OpenSora and Wan 2.2 5B on our newly constructed large-scale benchmark, NewtonBench-60K. Across both models in visual and physics metrics, NewtonRewards consistently improves physical plausibility, motion smoothness, and temporal coherence over prior post-training methods. It further maintains strong performance across all primitives and under out-of-distribution shifts in height, speed, and friction. Our results show that physics-grounded verifiable rewards offer a scalable path toward physics-aware video generation.

Primary area:
generative models
Keywords:
Post-Training, Video Generation, Verifiable Rewards

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_s48zOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces NewtonRewards, a physics-grounded post-training framework for improving the physical realism of video generation. Its key idea is to convert known physical laws into verifiable, differentiable rewards using measurable proxies extracted from generated videos: optical flow is used as a proxy for velocity to enforce constant-acceleration Newtonian dynamics, while V-JEPA features provide a complementary physical-properties consistency reward that prevents degenerate solutions such as vanishing or static objects. The method is evaluated on five Newtonian motion primitives using the proposed NewtonBench-60K dataset, and consistently improves both visual quality and physical consistency on OpenSora and Wan 2.2.

Critical strengths

A key strength of this paper is that it targets an important and fundamental limitation of current video generation models: the lack of reliable physical consistency. Improving adherence to basic physical laws is highly relevant for developing more realistic and trustworthy video generators and world models.

Critical weaknesses
  1. The claimed scalability and generality of the proposed post-training paradigm remain unclear. All five motion primitives considered in this work are deliberately chosen such that they can be reduced to essentially the same constant-acceleration constraint. In more realistic scenarios involving collisions, contacts, articulated or deformable objects, time-varying forces, or complex interactions, one would need to manually identify suitable physical laws and derive reliable measurable proxies for each case. Therefore, the current experiments demonstrate the effectiveness of one particular kinematic constraint, but do not yet provide strong evidence that the framework can scale toward generally physics-aware video generation.
  2. The technical novelty also appears somewhat incremental relative to the rapidly growing literature on physics-guided video generation and post-training. Prior works such as PhysGen and PhysCtrl already leverage simulated physical dynamics to guide generation, while PISA performs physics-oriented post-training using optical-flow, depth, and segmentation rewards. More recently, PhysRVG also directly optimizes video generators with physics-aware rewards. The main distinction here is to replace reference-based supervision with a manually specified equation-of-motion residual. While this formulation is clean, constructing a constant-acceleration loss from optical flow appears relatively straightforward, and I would like to see a clearer demonstration of what fundamentally new capability this formulation enables beyond existing simulation- or reward-guided approaches.
  3. The empirical validation remains rather limited. Although the appendix includes an evaluation on real-world free-fall videos, the main training and benchmark are entirely based on controlled synthetic scenes, and even the real-world experiment is restricted to the same simple free-fall setting. More importantly, it is unclear whether post-training on a relatively narrow synthetic physics distribution preserves the original generator's general-purpose capabilities. Evaluations on diverse natural prompts and standard video-generation metrics would be important to rule out distribution shift or catastrophic degradation in visual quality, diversity, and prompt adherence.
  4. The use of optical flow as a velocity proxy relies on rather restrictive assumptions. The current setup uses a static camera, static background, and approximately planar object motion, under which image-space flow can reasonably reflect translational velocity. In realistic videos, however, camera motion, depth variation, occlusion, deformation, and object rotation can introduce substantial flow components unrelated to the center-of-mass dynamics being modeled. In particular, rotational rigid-body motion does not generally satisfy the proposed per-pixel constant-acceleration residual. It is therefore unclear how robust the proposed reward remains once these controlled assumptions are relaxed.
Questions for authors

Refer to the weakness.

1 comment
AuthorsOpenReview ↗

Response to Reviewer s48z

Thank you for the review. We agree that the paper should separate the demonstrated contribution from the broader goal of generally physics-aware generation.

1. Generality and scalability. The shared constraint across the five primitives is intentional: it tests whether one reward can improve several motion regimes without a separate objective for each. Experiments on two generators establish cross-backbone applicability, while the OpenSora OOD results test held-out physical parameters. They do not establish coverage of arbitrary interactions or automatic discovery of suitable proxies. Extending the approach to new dynamics would require additional constraints and reliable measurements. We will distinguish this research direction from the capabilities actually demonstrated.

2. What the formulation adds. One clarification is important: NewtonRewards adds a kinematic regularizer to reference supervision; it does not replace reference supervision. PhysGen and PhysCtrl use simulated or learned dynamics to guide generation. PhysRVG directly post-trains generators using reference-trajectory agreement. Our kinematic term instead evaluates a temporal relation within the generated sequence, complementing the reference-based anchor.

The relevant evidence is therefore the controlled ablation: with paired supervision unchanged, adding the kinematic term raises average improvement from 0.25% to 9.75% on OpenSora and 0.93% to 13.56% on Wan. This demonstrates useful training information beyond reference alignment alone. We do not claim novelty in the finite-difference operator itself, or superiority over every physics-guided system. The contribution is its effective use for generator post-training, the identification of its degenerate solutions, and the complementary objective that mitigates them.

3. Preservation of general-purpose capabilities. This remains an unanswered question. The 361-video real-world experiment supports simulation-to-real transfer for free fall; it does not establish preservation of diversity, prompt adherence, or performance on unrestricted natural prompts. Likewise, improved benchmark metrics cannot rule out forgetting elsewhere. Our evidence supports task-specific improvements under matched training conditions, not an unconditional upgrade to the generator’s general capabilities. We will make that limitation explicit.

4. Optical flow and rotation. You are correct that dense optical flow is not generally equivalent to center-of-mass velocity. Fixed-camera and planar-motion assumptions reduce confounding but do not eliminate rotation, occlusion, deformation, or changing object boundaries. Consequently, physically valid rigid-body motion can violate the per-pixel residual.

We will describe it as an approximate, physics-motivated image-space regularizer rather than an exact test of rigid-body dynamics. Its demonstrated usefulness rests on improved centroid-based velocity and acceleration errors against reference trajectories, measured separately from the flow residual optimized during training. That supports effectiveness within the tested regime, without establishing robustness once those assumptions are relaxed.

Reviewer_4GeBOpenReview ↗
Rating
1(−1)
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces NewtonRewards, a physics-grounded post-training framework for video generation. It leverages frozen models to extract proxy quantities of physical information from generated videos, using optical flow as a proxy for velocity and high-level visual features as proxies for physical properties. These proxies are employed to construct a reward function for model optimization. Meanwhile, the authors build NewtonBench-60K, a large-scale benchmark dataset covering five categories of Newtonian motion primitives, and fine-tune OpenSora and Wan 2.2 5B. Experiments demonstrate that NewtonRewards yields measurable improvements in enhancing physical plausibility.

Critical strengths
  • The problem the authors seek to address is highly significant. Nearly all video generation models struggle to satisfy physical consistency, which is also critical across a wide range of application scenarios.

  • The authors conduct extensive experiments across two generative models, five categories of motions, and both in-distribution (ID) and out-of-distribution (OOD) settings.

  • The two complementary rewards proposed by the authors optimize motion residuals while mitigating reward hacking artifacts such as object disappearance or static objects.

Critical weaknesses
  • The velocity proxy does not necessarily correspond to the correct physical law. The method only enforces ϕt+1−2ϕt+ϕt−1=0\phi_{t+1}-2\phi_t+\phi_{t-1}=0. Consequently, physically incorrect motions—such as an object accelerating upward during free fall or exhibiting constant horizontal acceleration in a horizontal-throw scenario—would still satisfy the constraint. At most, this reward enforces the form of constant-acceleration motion. Referring to it as a “Newtonian constraint” therefore appears to be an overclaim.

  • The physical-properties proxy effectively compares the features of each generated frame with those of a paired simulated reference frame. This comparison may simultaneously supervise object appearance, position, pose, and other visual attributes. Thus, the observed improvements may result from imitating the reference video rather than from conserving physical properties.

  • The methodological novelty appears limited. The two main components are both relatively standard techniques. The paper mainly combines them and interprets the resulting objective through a Newtonian lens, but it does not introduce a fundamentally new physical representation. The distinction from existing optical-flow consistency, temporal smoothness, and reference-feature alignment methods should therefore be clarified and supported by stronger comparisons.

Questions for authors
  • Do the paired reference features leak positional and motion supervision? How does performance change when the reference videos are randomly shuffled?

  • Can the V-JEPA reward distinguish different masses when object appearance is held fixed?

2 comments
AuthorsOpenReview ↗

Response to Reviewer 4GeB

Thank you for identifying where our interpretation goes beyond what the objective establishes. We clarify the constraint, the reference supervision, and the contribution below.

1. What the kinematic reward enforces. Your counterexamples are valid: the residual does not determine acceleration’s direction or magnitude. It is an optical-flow-based constant-acceleration regularizer, not a complete verifier of Newton’s laws. We will correct that wording. Its role is to constrain temporal structure within conditional, reference-guided training; scene-specific supervision comes from the training data. The fact that the residual is insufficient alone does not make it uninformative—the relevant empirical question is whether adding it improves motion accuracy under otherwise unchanged supervision.

2. Reference features and shuffling. Yes, paired features can carry position, pose, and motion information. We should therefore describe the feature term as a reference-based consistency anchor, rather than a disentangled measurement of physical properties.

However, reference alignment alone does not explain the observed gains. Keeping the same paired supervision while removing the kinematic term reduces average improvement from 9.75% to 0.25% on OpenSora, and from 13.56% to 0.93% on Wan. This establishes the additional value of the motion constraint; it does not establish that reference imitation is absent.

We have not run the shuffled-reference experiment. It would test dependence on correct pairing, although shuffling changes both appearance and motion targets and would not, by itself, separate their contributions.

3. Mass with appearance held fixed. We have not established that V-JEPA distinguishes mass under this control. The reported correlation is insufficient to support that interpretation, and we will qualify the claim. There is also an identifiability issue in our setting: mass cancels from ideal free-fall and frictional-sliding acceleration. With appearance, initial conditions, and other parameters fixed, different masses can produce identical videos. Our method therefore neither requires nor demonstrates a visual mass estimator.

4. Distinction and novelty. The contribution is a post-training objective and its demonstrated utility, not a new physical representation or finite-difference operator. The distinction from first-difference flow smoothing is precise: that penalty favors constant velocity, whereas our second-difference penalty permits nonzero constant acceleration. Unlike reference-flow matching, it evaluates a relation within the generated sequence rather than agreement with a prescribed flow field.

We agree that this remains a higher-order temporal regularizer. Our contribution is using this physically motivated prior as differentiable feedback for video-generator post-training, pairing it with an anchor against degenerate solutions, and demonstrating its benefit across two generators and five motion regimes. The ablations support that contribution; they do not establish superiority over every alternative temporal regularizer.

What exactly is this rebuttal trying to accomplish? In particular, what do the authors mean by:

“We have not run the shuffled-reference experiment. It would test dependence on correct pairing, although shuffling changes both appearance and motion targets and would not, by itself, separate their contributions”?

This statement reads like an AI-generated, defensive response rather than a genuine scientific clarification. I am seriously concerned that this rebuttal may not have received meaningful human scrutiny.

More importantly, the rebuttal does not address my core concerns or provide convincing evidence to resolve them. I have decided to lower my score.

Reviewer_1sgAOpenReview ↗
Rating
1(−1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper proposes NewtonRewards, a post-training method for physically plausible video generation. It penalizes the second-order temporal difference of optical flow to enforce constant acceleration. A V-JEPA 2 feature-matching term against simulated reference videos keeps the model from simply freezing the object. The authors also build NewtonBench-60K, a simulated dataset of five Newtonian motion types.

Critical strengths

1.The problem is meaningful. NewtonBench-60K, with five motion types and ID/OOD splits, is a useful resource. 2.The method is simple and intuitive, and the physics-based reward is sound for the motions studied. 3.Results beat the compared baselines on average, and some qualitative examples, such as the ramp scenes, look smoother.

Critical weaknesses

1.Recent simulation-guided methods (3DPhysVideo, PhysAgent, NEWTON) are neither cited nor compared. Methods the paper discusses in Appendix A (PhysGen, NewtonGen, PhysRVG) are also missing from the experiments, and the submitted video examples look weaker to me than those of these works. 2.The feature term is reference alignment, similar to the PISA rewards, and the method collapses without it, so it relies on simulated reference videos. 3.The method is limited to a narrow scope...The loss assumes a fixed camera, a static background and constant forces, so contacts, collisions and camera motion are not handled. The real-world test covers only free fall. 4.There is no FVD or VBench score, no human study.

Questions for authors

1.Can you compare with physics-based methods, especially simulation-guided ones (3DPhysVideo, PhysAgent, NEWTON) and those in Appendix A? If the inputs differ, a discussion of the trade-offs would help. 2.Can the method work without simulated reference videos, for example on real data?

2 comments
AuthorsOpenReview ↗

Response to Reviewer 1sgA

Thank you for recognizing the benchmark’s value and the method’s effectiveness within the studied motions. We clarify four points.

1. Comparison scope. We agree that the missing works should be cited. 3DPhysVideo, PhysAgent, NEWTON, PhysGen, and NewtonGen introduce simulation, dynamics models, or agentic control during generation. These provide richer physical control; NewtonRewards instead updates the generator itself, without requiring a simulator or planner at inference. Our matched experiments isolate the effect of reward formulation, rather than establish superiority over these complete systems.

PhysRVG is a closer comparator: it optimizes reference-trajectory agreement, whereas our kinematic reward tests a motion relation within the generated sequence. A direct comparison remains valuable; the methodological distinction alone does not establish superiority.

2. Is the gain simply reference alignment? The existing ablation directly tests this explanation. Keeping reference-feature alignment but removing the kinematic term yields only 0.25%/0.93% average improvement on OpenSora/Wan, versus 9.75%/13.56% with the full objective. Both variants use the same references. Thus, reference alignment alone does not explain the gains.

The reference term nevertheless has a necessary role in the proposed recipe: stationary scenes can also minimize the kinematic residual, and the feature anchor discourages this degeneracy. We do not claim reference-free training. At inference, however, no reference future video is needed. Real videos could supply training references in principle, but this remains untested. Our 361-video real-world experiment demonstrates transfer from simulation-trained models, not real-data post-training.

3. Physical scope. Fixed-camera and constant-force assumptions are genuine limitations. However, “contacts are not handled” is too broad: ramp sliding includes sustained contact and friction. Impulsive collisions, contact transitions, and moving cameras remain outside the demonstrated scope. The five primitives and OOD parameter shifts test generalization within these regimes; they do not establish unrestricted physical understanding. The real-world evidence is likewise limited to free fall.

4. Evaluation. FVD, VBench, and human judgments would complement our measurements; their absence limits claims about overall perceptual superiority. Our central evidence concerns motion accuracy: velocity and acceleration errors are measured from object-mask centroids against reference trajectories, not from the optical-flow residual used for training. The improvements therefore reflect more than optimizing the training score. Together with the controlled ablations, they support the specific contribution: explicit kinematic constraints improve generated dynamics beyond reference alignment alone.

I did not see anything new in the rebuttal. There are no new experiments for justification. The response also agrees with most of the main criticisms. The core concerns are not resolved, so I am lowering my rating.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.