- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces NewtonRewards, a physics-grounded post-training framework for improving the physical realism of video generation. Its key idea is to convert known physical laws into verifiable, differentiable rewards using measurable proxies extracted from generated videos: optical flow is used as a proxy for velocity to enforce constant-acceleration Newtonian dynamics, while V-JEPA features provide a complementary physical-properties consistency reward that prevents degenerate solutions such as vanishing or static objects. The method is evaluated on five Newtonian motion primitives using the proposed NewtonBench-60K dataset, and consistently improves both visual quality and physical consistency on OpenSora and Wan 2.2.
Critical strengths
A key strength of this paper is that it targets an important and fundamental limitation of current video generation models: the lack of reliable physical consistency. Improving adherence to basic physical laws is highly relevant for developing more realistic and trustworthy video generators and world models.
Critical weaknesses
- The claimed scalability and generality of the proposed post-training paradigm remain unclear. All five motion primitives considered in this work are deliberately chosen such that they can be reduced to essentially the same constant-acceleration constraint. In more realistic scenarios involving collisions, contacts, articulated or deformable objects, time-varying forces, or complex interactions, one would need to manually identify suitable physical laws and derive reliable measurable proxies for each case. Therefore, the current experiments demonstrate the effectiveness of one particular kinematic constraint, but do not yet provide strong evidence that the framework can scale toward generally physics-aware video generation.
- The technical novelty also appears somewhat incremental relative to the rapidly growing literature on physics-guided video generation and post-training. Prior works such as PhysGen and PhysCtrl already leverage simulated physical dynamics to guide generation, while PISA performs physics-oriented post-training using optical-flow, depth, and segmentation rewards. More recently, PhysRVG also directly optimizes video generators with physics-aware rewards. The main distinction here is to replace reference-based supervision with a manually specified equation-of-motion residual. While this formulation is clean, constructing a constant-acceleration loss from optical flow appears relatively straightforward, and I would like to see a clearer demonstration of what fundamentally new capability this formulation enables beyond existing simulation- or reward-guided approaches.
- The empirical validation remains rather limited. Although the appendix includes an evaluation on real-world free-fall videos, the main training and benchmark are entirely based on controlled synthetic scenes, and even the real-world experiment is restricted to the same simple free-fall setting. More importantly, it is unclear whether post-training on a relatively narrow synthetic physics distribution preserves the original generator's general-purpose capabilities. Evaluations on diverse natural prompts and standard video-generation metrics would be important to rule out distribution shift or catastrophic degradation in visual quality, diversity, and prompt adherence.
- The use of optical flow as a velocity proxy relies on rather restrictive assumptions. The current setup uses a static camera, static background, and approximately planar object motion, under which image-space flow can reasonably reflect translational velocity. In realistic videos, however, camera motion, depth variation, occlusion, deformation, and object rotation can introduce substantial flow components unrelated to the center-of-mass dynamics being modeled. In particular, rotational rigid-body motion does not generally satisfy the proposed per-pixel constant-acceleration residual. It is therefore unclear how robust the proposed reward remains once these controlled assumptions are relaxed.
Questions for authors
Refer to the weakness.
Response to Reviewer s48z
Thank you for the review. We agree that the paper should separate the demonstrated contribution from the broader goal of generally physics-aware generation.
1. Generality and scalability. The shared constraint across the five primitives is intentional: it tests whether one reward can improve several motion regimes without a separate objective for each. Experiments on two generators establish cross-backbone applicability, while the OpenSora OOD results test held-out physical parameters. They do not establish coverage of arbitrary interactions or automatic discovery of suitable proxies. Extending the approach to new dynamics would require additional constraints and reliable measurements. We will distinguish this research direction from the capabilities actually demonstrated.
2. What the formulation adds. One clarification is important: NewtonRewards adds a kinematic regularizer to reference supervision; it does not replace reference supervision. PhysGen and PhysCtrl use simulated or learned dynamics to guide generation. PhysRVG directly post-trains generators using reference-trajectory agreement. Our kinematic term instead evaluates a temporal relation within the generated sequence, complementing the reference-based anchor.
The relevant evidence is therefore the controlled ablation: with paired supervision unchanged, adding the kinematic term raises average improvement from 0.25% to 9.75% on OpenSora and 0.93% to 13.56% on Wan. This demonstrates useful training information beyond reference alignment alone. We do not claim novelty in the finite-difference operator itself, or superiority over every physics-guided system. The contribution is its effective use for generator post-training, the identification of its degenerate solutions, and the complementary objective that mitigates them.
3. Preservation of general-purpose capabilities. This remains an unanswered question. The 361-video real-world experiment supports simulation-to-real transfer for free fall; it does not establish preservation of diversity, prompt adherence, or performance on unrestricted natural prompts. Likewise, improved benchmark metrics cannot rule out forgetting elsewhere. Our evidence supports task-specific improvements under matched training conditions, not an unconditional upgrade to the generator’s general capabilities. We will make that limitation explicit.
4. Optical flow and rotation. You are correct that dense optical flow is not generally equivalent to center-of-mass velocity. Fixed-camera and planar-motion assumptions reduce confounding but do not eliminate rotation, occlusion, deformation, or changing object boundaries. Consequently, physically valid rigid-body motion can violate the per-pixel residual.
We will describe it as an approximate, physics-motivated image-space regularizer rather than an exact test of rigid-body dynamics. Its demonstrated usefulness rests on improved centroid-based velocity and acceleration errors against reference trajectories, measured separately from the flow residual optimized during training. That supports effectiveness within the tested regime, without establishing robustness once those assumptions are relaxed.