- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper shows that post-training video generation modes with verifiable rewards don't always optimize the intended physics properties. They fine-tune video generated with kinematic residual and optical rewards and show that these two results are highly correlated. Then they show that verifiable rewards lead to disagreement with human ratings from a VLM physics judge. This paper doesn't include any real human evaluation.
Critical strengths
This paper carefully analyses whether post-training video generators with verifiable rewards actually optimize the intended physics quantity. This is an interesting question that the post-training community should care about when they design verifiable rewards.
Critical weaknesses
The writing is very, very hard to follow, which I believe it's AI writing style. The paper starts directly with quantitative results without introducing the context and motivation carefully. Text is broken and not easy for humans to read.
There is no label in all the tables, and I cannot keep track of what they are talking about.
In NewtonRewards, they already mentioned that optimizing only kinematic residual rewards can lead to motion suppression and reward hacking (static motion and object disappearance).
This paper later uses VLMs trained on human ratings as a judge, but this evaluation setting is not reliable enough and cannot show whether it actually follows physics. So the conclusion can be that optimizing the physics verifiable rewards can reduce agreement in human judgment.
Questions for authors
The authors should discuss other physics verifiable rewards; this paper only analyzes the motion reward, and the insight doesn't show whether it applies to other physical laws.
I think the status of this paper is not ready for publication. I like the idea and intuition from this paper and would recommend that the authors revise this work heavily for the next conference cycle.
Response to Reviewer mvtt
Thank you for the review and for describing the question as one the post-training community should care about. Several statements in the review describe the paper differently from what it does. We take this as a sign that our writing was unclear, so we clarify those first and then answer the remaining points.
"This paper doesn't include any real human evaluation."
Not quite. The audit is built on human ratings, 3350 rated clips from seven generators in VideoPhy-2 and 9569 clips from five generators in Physion-Eval. What the paper lacks is human ratings of our own post-trained videos, which the revision states as a limitation (general response, point 3).
"This paper later uses VLMs trained on human ratings as a judge, but this evaluation setting is not reliable enough and cannot show whether it actually follows physics."
Partly. The judge is not ground truth, and Section 4.5 now says so. It is, however, never the yardstick in our audit. Every signal, the rewards and the judge alike, is measured against human ratings, and the judge is a second object of the same test. On VideoPhy-2, once peak flow and mean velocity are controlled, it keeps 88.0% of its agreement with human ratings, against −1.7% for the residual.
"So the conclusion can be that optimizing the physics verifiable rewards can reduce agreement in human judgment."
We do not draw this conclusion either. We have no human ratings of optimized videos to test it. Our conclusion is narrower. Once motion is controlled, the audited reward retains an association with human ratings in at most one of 12 generators.
"They fine-tune video generated with kinematic residual and optical rewards and show that these two results are highly correlated."
Not quite. The high correlation is between the residual reward and peak optical flow, which is a motion statistic rather than a second reward. It holds in all five sources of clips, which cover 13 generators and include both human-rated datasets.
"The authors should discuss other physics verifiable rewards; this paper only analyzes the motion reward…"
Partly. The paper analyses two rewards, and only one of them is tied to motion. PhyMotion recovers 3D human bodies and scores them in a physics simulator. Its correlation with peak flow is 0.18, against 0.92 for the residual on the same clips (Section 4.4). Several other published rewards need a reference video and cannot be computed on these generations (Section 5). We agree that two rewards do not show how the finding extends to other physical laws, and the revised title and abstract are limited accordingly.
"In NewtonRewards, they already mentioned that optimizing only kinematic residual rewards can lead to motion suppression and reward hacking (static motion and object disappearance)."
We agree and now cite their Finding 2 and Table 4 (general response, point 1). On object disappearance, our object check finds it clearly raised in one of three residual runs and in two of three placebo runs, so in our runs it does not follow the reward.
"The paper starts directly with quantitative results without introducing the context and motivation carefully."
We agree. The abstract is rewritten so that it gives the context and the question before any number. We also removed repeated caveats and gave the audited reward one name throughout. A fuller rewrite of the main text is planned for the next version.
"There is no label in all the tables…"
We agree. All tables are now numbered and captioned (Tables 1 to 9).
We take the recommendation for a substantial revision seriously. We hope these answers address your concerns, and we are happy to discuss any point further.