acceptodds
Under review as a conference paper at ICLR 2027

Verifiable ≠ Valid: What Programmatic Physics Rewards for Video Generation Actually Optimize

Abstract

Post-training video generators with verifiable physics rewards, meaning programmatic quantities read deterministically off frozen perception models, has become a leading alternative to VLM judges. Such rewards are validated by showing that they separate real video from rule-corrupted video. We find that this validation systematically overstates their validity, that what the two rewards we could measure actually optimize is motion magnitude, and that removing the motion confound is necessary but far from sufficient. Across two human-rated datasets, 13 generators and two architectures, a reimplemented kinematic-residual reward correlates with peak optical flow at –. Once motion is partialled out it retains no association with human ratings in 0 of 12 generators under a single motion control, and a pure motion statistic containing no physics predicts human ratings better than it does. Optimization reproduces this causally: in two GRPO runs identical but for the reward, the motion-correlated reward drives peak flow down 46% (95% CI ) while a motion-orthogonal reward drives it up 67%, the arms separating monotonically in dose at , with three motion-orthogonal signals pushed in no consistent direction; a lower-fidelity second prompt set replicates it, with peak flow down 88% at the highest dose and the arms separating at all five doses. The identical test applied to a VLM physics judge answers oppositely: the judge retains 88.0% of its agreement with human ratings once motion is partialled out, against % for the programmatic reward. The dividing line is therefore not verifiability but whether the quantity is a motion statistic. Escaping the confound is not sufficient either: the most structured physics reward published to date, which grounds recovered 3D bodies in a physics simulator, does escape it ( with motion, against ) yet, tested outside its design domain, carries no measurable validity beyond motion on this rating (partial , ). We stop short of the last step. We do not show that the invalid reward makes videos worse for people: we have no human ratings on our own generations, the two runs disagree about the automatic judge, and treating a rising automatic physics score as evidence about human judgement is the very inference this paper is written against.

Primary area:
datasets and benchmarks
Keywords:
physics rewards, video generation, reward validity, motion confound, evaluation of evaluators, VLM judges

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_mvttOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper shows that post-training video generation modes with verifiable rewards don't always optimize the intended physics properties. They fine-tune video generated with kinematic residual and optical rewards and show that these two results are highly correlated. Then they show that verifiable rewards lead to disagreement with human ratings from a VLM physics judge. This paper doesn't include any real human evaluation.

Critical strengths

This paper carefully analyses whether post-training video generators with verifiable rewards actually optimize the intended physics quantity. This is an interesting question that the post-training community should care about when they design verifiable rewards.

Critical weaknesses

The writing is very, very hard to follow, which I believe it's AI writing style. The paper starts directly with quantitative results without introducing the context and motivation carefully. Text is broken and not easy for humans to read.

There is no label in all the tables, and I cannot keep track of what they are talking about.

In NewtonRewards, they already mentioned that optimizing only kinematic residual rewards can lead to motion suppression and reward hacking (static motion and object disappearance).

This paper later uses VLMs trained on human ratings as a judge, but this evaluation setting is not reliable enough and cannot show whether it actually follows physics. So the conclusion can be that optimizing the physics verifiable rewards can reduce agreement in human judgment.

Questions for authors

The authors should discuss other physics verifiable rewards; this paper only analyzes the motion reward, and the insight doesn't show whether it applies to other physical laws.

I think the status of this paper is not ready for publication. I like the idea and intuition from this paper and would recommend that the authors revise this work heavily for the next conference cycle.

1 comment
AuthorsOpenReview ↗

Response to Reviewer mvtt

Thank you for the review and for describing the question as one the post-training community should care about. Several statements in the review describe the paper differently from what it does. We take this as a sign that our writing was unclear, so we clarify those first and then answer the remaining points.

"This paper doesn't include any real human evaluation."

Not quite. The audit is built on human ratings, 3350 rated clips from seven generators in VideoPhy-2 and 9569 clips from five generators in Physion-Eval. What the paper lacks is human ratings of our own post-trained videos, which the revision states as a limitation (general response, point 3).

"This paper later uses VLMs trained on human ratings as a judge, but this evaluation setting is not reliable enough and cannot show whether it actually follows physics."

Partly. The judge is not ground truth, and Section 4.5 now says so. It is, however, never the yardstick in our audit. Every signal, the rewards and the judge alike, is measured against human ratings, and the judge is a second object of the same test. On VideoPhy-2, once peak flow and mean velocity are controlled, it keeps 88.0% of its agreement with human ratings, against −1.7% for the residual.

"So the conclusion can be that optimizing the physics verifiable rewards can reduce agreement in human judgment."

We do not draw this conclusion either. We have no human ratings of optimized videos to test it. Our conclusion is narrower. Once motion is controlled, the audited reward retains an association with human ratings in at most one of 12 generators.

"They fine-tune video generated with kinematic residual and optical rewards and show that these two results are highly correlated."

Not quite. The high correlation is between the residual reward and peak optical flow, which is a motion statistic rather than a second reward. It holds in all five sources of clips, which cover 13 generators and include both human-rated datasets.

"The authors should discuss other physics verifiable rewards; this paper only analyzes the motion reward…"

Partly. The paper analyses two rewards, and only one of them is tied to motion. PhyMotion recovers 3D human bodies and scores them in a physics simulator. Its correlation with peak flow is 0.18, against 0.92 for the residual on the same clips (Section 4.4). Several other published rewards need a reference video and cannot be computed on these generations (Section 5). We agree that two rewards do not show how the finding extends to other physical laws, and the revised title and abstract are limited accordingly.

"In NewtonRewards, they already mentioned that optimizing only kinematic residual rewards can lead to motion suppression and reward hacking (static motion and object disappearance)."

We agree and now cite their Finding 2 and Table 4 (general response, point 1). On object disappearance, our object check finds it clearly raised in one of three residual runs and in two of three placebo runs, so in our runs it does not follow the reward.

"The paper starts directly with quantitative results without introducing the context and motivation carefully."

We agree. The abstract is rewritten so that it gives the context and the question before any number. We also removed repeated caveats and gave the audited reward one name throughout. A fuller rewrite of the main text is planned for the next version.

"There is no label in all the tables…"

We agree. All tables are now numbered and captioned (Tables 1 to 9).

We take the recommendation for a substantial revision seriously. We hope these answers address your concerns, and we are happy to discuss any point further.

Reviewer_SCFcOpenReview ↗
Rating
2(+1)
Confidence
4
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The paper investigates whether programmatic physics rewards primarily measure motion magnitude. It combines motion-controlled correlations on two human-rated datasets with paired GRPO interventions and a VLM-judge comparison, finding strong motion dependence for a reimplemented kinematic residual.

Critical strengths

The results on two datasets show that the tested reward is closely tied to the amount of motion in a video. Using the same prompts and random seeds, the authors show that training with this reward makes the generated videos move less (§4.3; Appendix D). The VLM comparison is also informative: its scores remain related to human ratings even after controlling for motion (§4.5).

Critical weaknesses
  1. The audit does not substantiate its novelty or broader scope. The intervention optimizes only NewtonRewards' kinematic residual (§3.4). Yet NewtonRewards already documents residual-only motion collapse and adds a mass-conservation term to mitigate it (§6.3; Appendix B). Testing the isolated component without that safeguard substantially weakens the claimed contribution. The second reward, PhyMotion, is evaluated against general physical-commonsense labels outside its intended human-motion task (§4.4; §5), despite existing in-domain human validation (§5.1). These cases support a limited audit, not the paper's broad framing of what programmatic physics rewards optimize.

  2. The validity criterion and intervention do not establish the claimed mechanism. §3.1 treats asymmetric partial correlations as licensing directionality. This does not identify causation; conditioning on motion can also remove legitimate physical information. The GRPO experiments establish motion suppression, but their area-based controls are not validated measures of physical correctness. §5 acknowledges both these signals' weak validity and the absence of human ratings on optimized outputs. Consequently, the experiments cannot distinguish genuine physical improvement from action omission. The paper's extensive caveats acknowledge this gap without resolving the central interpretation.

  3. The writing and presentation need substantial revision. Frequent shifts between different rewards and repeated caveats make the main argument difficult to follow. Some definitions and figure descriptions are also inconsistent: Figure 1 states that “only the reward differs,” although it compares CogVideoX with VideoREPA. These inconsistencies make it harder to understand what was measured and how the comparisons were constructed. The paper needs clearer organization and a careful check of its definitions, claims, and figure labels.

Questions for authors
  1. Validation of current claims: Does the diagnosis persist for the complete NewtonRewards objective and for PhyMotion on its intended human-motion labels?
  2. Clarification: What identifies motion as a confound rather than a legitimate physical signal, and what evidence on optimized clips distinguishes action omission from physical improvement?
4 comments
AuthorsOpenReview ↗

Response to Reviewer SCFc (1/2)

Thank you for a careful and accurate review, and in particular for pointing us to the NewtonRewards result. We are glad you found the two-dataset result and the VLM comparison informative. We agree with much of the review and answer every point below, the largest first, in two comments.

"The audit does not substantiate its novelty or broader scope."

Partly. On scope we agree, and the next point says how the revision narrows it. On novelty we see it differently. NewtonRewards shows that optimizing the residual alone collapses motion, but it does not compare the residual with human ratings. Our audit does, on two human-rated datasets covering 12 generators. Once motion is controlled, at most one of the 12 retains any association between the residual and human ratings.

"These cases support a limited audit, not the paper's broad framing of what programmatic physics rewards optimize."

We agree. The title now reads "Verifiable ≠ Valid: Auditing Physics Rewards for Video Generation Beyond Motion", and the abstract and introduction speak only about the rewards we measured.

"Testing the isolated component without that safeguard substantially weakens the claimed contribution."

Partly. It weakens the post-training experiment, which we no longer present as new (general response, point 1). It does not weaken the audit. The residual is the part of NewtonRewards that can be computed on human-rated clips, and the audit asks a question that applies to any reward, whether it says anything about human ratings beyond the amount of motion. For the residual the answer is no in at least 11 of 12 generators, and that answer does not depend on the mass term.

"What identifies motion as a confound rather than a legitimate physical signal…"

Nothing does, and our argument does not need it to. Human ratings themselves change with motion, so motion carries real information about them, and the revision drops the confound framing. Whatever motion's status, a reward named for physics should tell us something about plausibility that the amount of motion does not already tell us. On VideoPhy-2, with peak flow and mean velocity controlled, the VLM judge does so and keeps 88.0% of its agreement with human ratings, while the residual keeps none.

"§3.1 treats asymmetric partial correlations as licensing directionality. This does not identify causation; conditioning on motion can also remove legitimate physical information."

We agree on both counts. We have withdrawn the directional reading, and Section 3.1 now states that partialling motion out can also remove legitimate physical information. The control does not remove everything, as the judge's result in the previous answer shows.

The answers continue in the next comment.

AuthorsOpenReview ↗

Response to Reviewer SCFc (2/2)

"…their area-based controls are not validated measures of physical correctness."

We agree. They serve only as controls that are verifiable in the same sense as the residual but are not motion statistics (Section 3.3), and the heading in Section 4.3 that called them physical signals is corrected. The paper itself reports that every such signal stays small after motion control, with |ρ| ≤ 0.15 on VideoPhy-2 (Section 5). The revision also drops the claim that the area-based arm optimized its own reward, since that reward did not move detectably.

"Consequently, the experiments cannot distinguish genuine physical improvement from action omission."

We agree, and we added a check. An open-vocabulary detector counts the clips in which the main object is missing or does not move (Appendix M, Tables 8 and 9). One of the three residual runs has clearly more such clips than the base model, 42 against 20. The submitted run does not, and two of three placebo runs do. No result indicates a physical improvement, so the revision reports only a drop in motion that cannot be attributed to the reward. The videos behind Figure 7 are in the new supplementary material.

"The paper's extensive caveats acknowledge this gap without resolving the central interpretation."

The new controls resolve it, against our original reading. The placebo shows that the drop in motion is not specific to the reward (general response, point 2), and the object check shows no physical improvement. The revision therefore states a single interpretation, a drop in motion that the reward does not explain, and removes the caveats that hedged between readings.

"The second reward, PhyMotion, is evaluated against general physical-commonsense labels outside its intended human-motion task…"

We agree. PhyMotion's own human study is not public, so we built a proxy closer to its domain. Among 1168 rated clips with a full body, we flagged the 86 whose annotated violation concerns the body or balance, using keywords fixed in advance. PhyMotion predicts this flag (ρ = 0.070, p = 0.016). Most of the association remains after motion control, although the test with both controls misses 0.05 (p = 0.030 with peak flow, p = 0.060 with both). We therefore limit our PhyMotion statement to general physical-commonsense ratings (Section 4.4, Appendix F).

"Does the diagnosis persist for the complete NewtonRewards objective and for PhyMotion on its intended human-motion labels?"

For the complete objective, not with the data available to us. Its mass term needs a simulated reference video for each clip (their Eq. 11). The human-rated datasets have none, so the audit cannot include it, and our text-to-video prompts have none, so we could not train with it there. NewtonBench-60K does provide such references and is public, but it has no human ratings, and the authors' training code is not released. Reimplementing the full training on NewtonBench-60K was beyond what we could do within the discussion period. For PhyMotion, the proxy above is the closest test available, because the human preferences from its own study are not released.

"Frequent shifts between different rewards and repeated caveats make the main argument difficult to follow. Some definitions and figure descriptions are also inconsistent: Figure 1 states that “only the reward differs,” although it compares CogVideoX with VideoREPA."

The reviewer is right. Figure 1 is relabelled "same prompt and seed, base model vs. VideoREPA". We also gave the audited reward one name throughout the text, corrected a mislabelled panel title in Figure 4 and two figure captions, and collected the limitations in Section 5. Our re-check of the claims led to the corrections listed in the summary of changes. Reducing the shifts between rewards needs a restructuring that we plan for the next version.

"…the authors show that training with this reward makes the generated videos move less…"

Partly. The submitted run does move less than the base model, but placebo runs move peak flow by −66% to +123% depending on the seed. The reduction therefore cannot be attributed to the reward (general response, point 2).

We hope these answers address your concerns, and we are happy to discuss any point further.

I appreciate the substantial additional experiments and raise my score to 2 (Weak rejection), but still recommend rejection given unresolved concerns about the interpretation of the results and the paper’s contribution. The AI-assisted writing needs a thorough rewrite: too many defensive caveats make the argument hard to follow and may push readers to rely on AI summaries.

AuthorsOpenReview ↗

Reply to Reviewer SCFc

Thank you for reading the new experiments and for reconsidering your score. Your review improved this paper directly. It pointed us to the NewtonRewards result that we had failed to credit, and it caught the mislabelled comparison in Figure 1. The object-level check and the PhyMotion test closer to its design domain both exist because you asked the questions behind them. We accept the recommendation as it stands, and we reply briefly because the discussion closes soon.

"…too many defensive caveats make the argument hard to follow…"

We agree. The revision removed caveats in several places but kept the structure of the submission, and the argument is still harder to follow than it should be. A proper rewrite needs a new outline in which each limitation is stated once. We cannot do that well in the hours that remain, so we will do it for the next version and will not upload a rushed one now.

"…unresolved concerns about the interpretation of the results and the paper’s contribution."

We want to address these in the next version, and it would help us to know which of two points weighs more for you. The first is the reading of the audit, in which at most one of 12 generators retains any association between the residual and human ratings once motion is controlled. The second is the contribution that remains once the post-training result is credited to NewtonRewards. If you have a moment before the discussion closes, one sentence on this would be very useful to us. If not, we understand. Either way, we are grateful for the time and care you put into this review.

Reviewer_zUYcOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors identify a less noticed problem in physics rewards for video generation: current works often validate physics rewards without verifying whether they truly reflect physical plausibility as perceived by humans. The authors find that the correlation between a kinematic-residual reward and human ratings largely disappears after controlling for motion magnitude. The authors further conduct experiments to support this finding.

Critical strengths

1.This paper raises an insightful point about physics rewards for video generation.

2.The authors conduct GRPO experiments and analyze the correlation between physics rewards and motion magnitude across multiple benchmarks and video generators.

Critical weaknesses

1.The abstract is very difficult to read and seems AI-generated. I think using AI is fine, but the abstract is not reader-friendly. I suggest rewriting it in a clearer and more readable way for the camera-ready version.

2.Since the main claim is related to human perception, I would like to see more qualitative results and human evaluation. Although the authors show that optimizing the physics reward can reduce motion magnitude, I am not sure how different the generated videos actually look to humans.

3.Motion is part of physics itself. If we control for motion magnitude, could we also remove some meaningful physics information?

Questions for authors
  1. Could the authors provide more qualitative results and human evaluation to address Weakness 2?

  2. Could the authors clarify the concern raised in Weakness 3 about whether controlling for motion magnitude may also remove meaningful physics information?

2 comments
AuthorsOpenReview ↗

Response to Reviewer zUYc

Thank you for the review and for finding the question insightful. We answer each point below, the largest first.

"Since the main claim is related to human perception, I would like to see more qualitative results and human evaluation."

We now provide the videos themselves, in place of a human study that we could not run. A human study with outside raters needs an ethics assessment, which we could not obtain within the discussion period. Instead, the new supplementary material contains the generated videos behind Figure 7, so that you can see how they differ. These are the same four held-out clips under all 11 conditions, 44 videos as generated and without re-encoding, with a page that plays them side by side. Figure 7 shows their middle frames. We also count, over all 80 clips per condition, how often the main object is missing or static (Tables 8 and 9). The revision states the missing ratings as the main limitation of the post-training experiment. The main claim itself is already tested against human perception, because the audit measures each reward against existing human ratings of 3350 VideoPhy-2 clips and 9569 Physion-Eval clips.

"If we control for motion magnitude, could we also remove some meaningful physics information?"

Yes, some of it, but not all. We agree this limits what the test shows. On VideoPhy-2, however, with peak flow and mean velocity controlled, a vision-language model (VLM) judge keeps 88.0% of its agreement with human ratings, so a signal that agrees with human ratings beyond motion does pass the test. For the kinematic residual, at most one of 12 generators retains any association once motion is controlled. The revision states the result in this narrower form (Section 3.1).

"The abstract is very difficult to read…"

We agree that it was hard to read, and have rewritten it. The new abstract states the problem before any number, keeps only the key results and states the main limitation once.

These answers also cover the two questions in the review. We hope they address your concerns, and we are happy to discuss any point further.

Thank you for the detailed response and additional experiments. I appreciate the authors' efforts. I will also consider other reviewers' opinions during the reviewer–AC discussion and revisit my assessment.

Comments on the paper

2 comments
AuthorsOpenReview ↗

General response to all reviewers and the AC

We thank the three reviewers for their careful reading. Reviewers zUYc and mvtt find the question worthwhile, namely whether a programmatic physics reward reflects physical plausibility as people judge it, and SCFc notes that the results on two datasets tie the tested reward closely to motion. The revised PDF is uploaded, and the next comment lists the changes.

What stands and what we withdraw. The submission rested on an audit and on a post-training experiment. The audit stands, and its results are unchanged. Across five sources of clips and 13 generators, including the two human-rated datasets, the reimplemented kinematic residual tracks peak optical flow closely. Its rank correlation with peak flow is ρ = 0.85–0.92 in every source. On VideoPhy-2, once peak flow and mean velocity are controlled, the residual keeps none of its agreement with human ratings (−1.7%), while a vision-language model (VLM) judge tested in the same way keeps 88.0%. Across both datasets, at most one of 12 generators retains any association between the residual and human ratings. We withdraw the claim that our post-training experiment shows the reward causing a drop in motion, because controls that we ran after submission do not support it (point 2 below). We also narrow one statement. The standard validation can overstate validity, rather than systematically does. The revised title and abstract present the paper as the audit, a test that any physics reward can be put through before it is used for training.

The reviewers' main concerns overlap, so we answer the shared ones here and reply to each reviewer below.

1. NewtonRewards already reports that optimizing the residual alone collapses motion (SCFc, mvtt). We agree, and we should have cited it. The revision credits this result in Sections 1, 2, 3.4 and 4.3, citing their Finding 2 and Table 4, and no longer presents our post-training result as new. We audit the residual alone because the full objective cannot be computed on human-rated clips. Its mass term needs a simulated reference video for each clip (their Eq. 11), and neither VideoPhy-2 nor Physion-Eval has one.

2. The post-training experiment does not show that the reward causes the drop in motion (SCFc). We agree, and new controls confirm it. In our post-training runs, a placebo reward unrelated to the video moves peak optical flow by −66% to +123% across three seeds. The residual arm ends below the placebo of its own seed in only one of them (Appendix M). We withdraw the causal wording and the directional reading of the partial correlations.

3. There are no human ratings of our own generated videos (zUYc, SCFc, mvtt). This is correct, and it is the main limitation of the post-training experiment, which the revision now states directly. A human study with outside raters needs an ethics assessment that we could not obtain within the discussion period. We therefore provide the generated videos themselves in the new supplementary material, together with a frame grid and an object-level count over all clips (Figure 7, Tables 8 and 9). The limitation does not affect the main result, which is measured against existing human ratings of 3350 VideoPhy-2 clips and 9569 Physion-Eval clips.

We recognise that these changes are substantial, and we understand if the reviewers and the AC prefer to assess the paper as submitted. Either way, we would value your view on whether the narrowed claims are now supported, since it will shape the next version. We are happy to discuss any point further.

AuthorsOpenReview ↗

Summary of changes in the revised PDF

  • The title is narrowed to "Verifiable ≠ Valid: Auditing Physics Rewards for Video Generation Beyond Motion", and the abstract is rewritten with the same findings.
  • Sections 1, 2, 3.4 and 4.3 credit NewtonRewards with the residual-only collapse, citing their Finding 2 and Table 4, and the causal and directional wording is removed.
  • Repeated caveats are removed from the introduction and results, and the limitations are collected in Section 5.
  • The audited reward is called the kinematic residual throughout the text, and VLM and BH are spelled out at first use.
  • Figure 1 is relabelled "same prompt and seed, base model vs. VideoREPA", a mislabelled panel title in Figure 4 is corrected, and all tables are numbered and captioned. The caption of Figure 2 no longer says that the reward moves the policy along a motion axis, and the caption of Figure 5 explains its bottom three rows.
  • The conclusion is narrowed to these data. It adds that a placebo reward and several seeds are needed before a change in a trained policy can be attributed to a reward.
  • New Appendix M reports the placebo at three seeds, a positive control, two further seeds of both arms and an object-detection check (Tables 6 to 9, Figure 7).
  • Appendix F adds a test of PhyMotion on body-related violations.
  • New supplementary material contains the 44 videos behind Figure 7, with a page that plays them side by side.
  • Corrections from our own re-check of the submission, each visible in the diff. Four correct a label or add a missing detail and leave the results unchanged. They concern the control sets of two partial correlations, the description of the VLM judge, the stratified reversal result and frame subsampling in 1032 stored residuals. Five narrow a statement. They concern the bound on motion-orthogonal signals for VideoPhy-2, the share of clips below the measured motion range, monotonicity in dose, arm B's own reward and the significance of the validation gap. On the last one, Section 4.1 and the abstract now say that the standard validation can overstate validity.
  • A final consistency pass corrects places where the text did not match our own tables and figures. The retention is now defined as the signed ratio that Table 3 reports. The abstract names the dataset and controls behind −1.7%. Appendix K now says that one of the twelve individually plotted generators, Veo 3.1 Fast at ρ = 0.81, lies below the 0.85–0.92 range. Section 4.2 quotes the Spearman value that matches its p-value, and Section 4.4 describes the PhyMotion proxy test as Appendix F does.

The main text stays within 10 pages.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.