- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper learns a physics reward for video-generator post-training. It constructs 10,114 preference pairs across 25 sub-scenes, using construction-based labels and rationales that identify and localize physical violations. These annotations train Phy-Judger, a VLM that explains its judgments and scores six physical dimensions. The judge, together with auxiliary quality rewards, guides joint RL of a prompt rewriter and MiniMax-H3. Validation includes held-out judge evaluation, component ablations, two physics benchmarks, and blinded human comparisons.
Critical strengths
The gains extend beyond the optimized judge. VBench Physics rises from 75.80 to 78.83, and PhyWorldBench physical commonsense rises from 0.155 to 0.203 (Tables 4–5, pp. 8–9). Human judgments also favor joint RL over the base model, 49.1% versus 33.1%, with randomized presentation, hidden method names, and matched seeds (Section 5.1, p. 6; Table 6, p. 9). Together, these results support an improvement in visible physical plausibility rather than merely a higher learned-reward score.
The annotation study offers useful practical evidence. In the 2,000-pair ablation, adding few-shot exemplars reduces dimension MAE from 1.42 to 0.46 (Table 1, p. 6). Appendix A.4 also groups shared prompts, videos, source clips, and near-duplicates within splits. These choices make the study of annotation quality more informative and the held-out evaluation more credible.
Critical weaknesses
The visual-grounding claim needs direct validation. Teachers receive the construction label and must make score orderings consistent with it; the label-consistency checker then reads only the rationale, not the videos (Appendix A.3, pp. 15–16). The UNSUPPORTED option is a meaningful safeguard, but textual consistency does not establish that a cited violation, timestamp, or box is correct. Tables 1–2 evaluate preferences and scores, not the accuracy of this localized evidence. This leaves a central claim—scores grounded in correctly identified visual violations—less well supported than the end-to-end scoring results. It is an evidence gap, not proof that the rationales are wrong.
The added value of preference-training the reward is not isolated. Tables 1–2 compare fine-tuned judges, and Table 3 compares policy-training variants, but none compares RL using Phy-Judger against its untuned Qwen3.5-9B backbone with the same rubric. The existing ablations therefore do not establish how much the new supervision improves downstream RL beyond pretrained VLM capabilities. Reward accounting also matters here: Appendix A.7 assigns a total weight of 0.3 to HPS/VideoAlign rewards, although Section 5.1 introduces these primarily as quality-tracking measures. Their gains should not be interpreted as a by-product of physical optimization alone. The independent physics improvements remain meaningful, but currently support the full recipe more strongly than the specific benefit of learning Phy-Judger.
Questions for authors
Clarification and targeted validation: Were the 200 held-out human preferences and dimension scores collected without revealing construction labels? A small label-blind audit of retained rationales, checking whether their violations and localizations match the videos, would directly address the grounding concern.
Clarification: Does Appendix A.7's 0.7/0.3 reward split apply to every RL variant in Table 3 and to both policies? Please state the complete objective and identify which evaluation metrics are also optimized.
Decision-relevant comparison: How does joint RL using untuned Qwen3.5-9B compare with Phy-Judger under identical rubrics, frame sampling, rollout budgets, and auxiliary rewards? A controlled advantage on the independent physics evaluations would establish the downstream value of the preference supervision.
6 comments
W2: RL results with untuned Qwen3.5-9B backbone with the same rubric. And single Phy-Judger reward.
- The added value of preference-training the reward is not isolated. Tables 1–2 compare fine-tuned judges, and Table 3 compares policy-training variants, but none compares RL using Phy-Judger against its untuned Qwen3.5-9B backbone with the same rubric. The existing ablations therefore do not establish how much the new supervision improves downstream RL beyond pretrained VLM capabilities. Reward accounting also matters here: Appendix A.7 assigns a total weight of 0.3 to HPS/VideoAlign rewards, although Section 5.1 introduces these primarily as quality-tracking measures. Their gains should not be interpreted as a by-product of physical optimization alone. The independent physics improvements remain meaningful, but currently support the full recipe more strongly than the specific benefit of learning Phy-Judger.
Thank you for this valuable suggestion. We add two RL variants under the same setting: (i) untuned Qwen3.5-9B + auxiliary rewards, which replaces Phy-Judger with its untuned backbone; and (ii) Phy-Judger only, without auxiliary rewards.
| Phy-Judger ↑ | VB-P ↑ | VB-C ↑ | MQ ↑ | TA ↑ | HPS-G ↑ | HPS-P ↑ | |
|---|---|---|---|---|---|---|---|
| MiniMax-H3 | 2.98 | 75.80 | 75.23 | 1.57 | 2.14 | 2.94 | 6.76 |
| (i) Untuned Qwen3.5-9B + auxiliary | 3.00 | 75.92 | 75.40 | 1.78 | 2.67 | 3.30 | 7.32 |
| (ii) Phy-Judger only | 3.70 | 78.90 | 78.53 | 1.50 | 2.25 | 2.60 | 6.81 |
| Phy-Judger + auxiliary (Ours) | 3.62 | 78.83 | 78.04 | 1.81 | 2.73 | 3.29 | 7.52 |
(i) The untuned backbone barely improve physics (+0.12 VB-P), since the untuned backbone is an unreliable reward: only 77.5% of its outputs are valid JSON, and its preference accuracy is 43.5% vs. 72.5% for Phy-Judger in the reward model test set. Under the same recipe, Phy-Judger adds +2.91 VB-P and +2.64 VB-C over it.
(ii) With Phy-Judger alone, RL achieves the full physics gain (+3.03 VB-P, +3.30 VB-C) without any auxiliary reward. Thus, essentially all of the physics gains come from the learned Phy-Judger. The 0.3 auxiliary rewards act as a standard RL regularizer against reward hacking: without them, the Phy-Judger score is the highest (3.70), but MQ and HPS-G drop below the base model.
We will revise Section 5.1 to clearly state that MQ, TA and HPS serve both as auxiliary rewards (total weight 0.3) and as general-quality metrics.
Q1: Clarification about the 200 held-out test pairs.
- Clarification and targeted validation: Were the 200 held-out human preferences and dimension scores collected without revealing construction labels? A small label-blind audit of retained rationales, checking whether their violations and localizations match the videos, would directly address the grounding concern.
Thank you for this helpful question. Yes. The 200 held-out pairs are the test set of the reward model. Their annotators see only the original prompt and the two videos, and give their own preference, per-dimension scores and rationale under the same rubric, without the construction labels.
The label-blind audit of the retained training rationales is reported in our response to W1. We further audit these held-out rationales in the same way: three new annotators first mark the time interval and bbox of each violation, then check whether each violation cited in the rationale is visible; tIoU / IoU are computed between their first-stage annotations and the cited violations confirmed visible. The agreement between every two new annotators serves as the upper bound.
| Comparison | Visible violation acc. ↑ | tIoU (time) ↑ | IoU (bbox) ↑ |
|---|---|---|---|
| Held-out rationale vs. new annotators | 92.5% | 0.67 | 0.61 |
| Inter-annotator (upper bound) | – | 0.67 | 0.62 |
92.5% of the cited violations are visible, and the localization nearly reaches the upper bound. Thus, the held-out annotations are reliable and grounded in visible violations.
Q2: Clarification about the reward setting.
- Clarification: Does Appendix A.7's 0.7/0.3 reward split apply to every RL variant in Table 3 and to both policies? Please state the complete objective and identify which evaluation metrics are also optimized.
Thank you for this careful question. Yes, the same 0.7/0.3 reward split is used for every RL variant in Table 3 and for both the video policy and the prompt policy. The complete objective is
where runs over the six Phy-Judger dimensions and over HPS-G, HPS-P, MQ and TA. is the Flow-GRPO objective (Eq. 2) with advantage , where averages over the videos of the same rewrite; is its token-level GRPO counterpart with advantage , where averages over the rewrites of the same prompt.
Among the metrics, Phy-Judger, MQ, TA, HPS-G and HPS-P are optimized, while VB-P, VB-C, PhyWorldBench and the human evaluation are not optimized and serve as independent evaluators. We will state this in Section 4.3 of the revision.
Q3: RL results with untuned Qwen3.5-9B backbone under identical settings.
- Decision-relevant comparison: How does joint RL using untuned Qwen3.5-9B compare with Phy-Judger under identical rubrics, frame sampling, rollout budgets, and auxiliary rewards? A controlled advantage on the independent physics evaluations would establish the downstream value of the preference supervision.
Thank you for this insightful question. Yes. Following your suggestion, we compare joint RL with Phy-Judger vs. its untuned Qwen3.5-9B backbone under identical settings: same rubric, frame sampling, rollout budget, and auxiliary rewards. The untuned run barely improves physics (+0.12 VB-P, +0.17 VB-C), while Phy-Judger adds +2.91 VB-P and +2.64 VB-C.
The detailed results are shown in W2. We hope this controlled comparison helps establish the value of the preference supervision.
We sincerely thank the reviewer for the detailed and constructive feedback. Below we address each of the raised concerns:
W1: Visual-grounding claim needs direct validation.
Thank you for this insightful suggestion. We directly validate the cited violations, timestamps and boxes in the rationales.
We sample 300 video pairs with their rationales, covering all three pair types (150 real–synthetic, 69 strong–weak and 81 correct–perturbed) and all annotation tiers. Three human experts, Gemini-4-Argon and Opus-5.5 first see only the videos, without the labels or rationales, and mark the time interval and bboxes of each physical violation (stage 1). They then see the rationale and check whether each cited violation is visible (stage 2).
For each annotator, we report the ratio of cited violations that are visible. tIoU / IoU measure the temporal / bounding-box overlap between each cited violation confirmed visible and the same violation marked by the annotator before seeing the rationale (stage 1). As an upper bound, we compute the same metrics between every two human experts and average over the three pairs.
87.7%–92.3% of the cited violations are visible, and the timestamps and boxes match human experts (tIoU 0.57–0.59, IoU 0.50–0.52) close to the inter-human agreement (0.65 / 0.60).
We also check the rationales generated by the trained Phy-Judger itself on the 200 held-out test pairs with the same protocol.
The training rationales are thus well grounded. The rationales of Phy-Judger are less precise but still largely grounded: 71.5% of the cited violations are visible, and its localization reaches over 70% of the inter-human upper bound, far above the untuned Qwen3.5-9B backbone evaluated with the same protocol (tIoU 0.23, IoU 0.20). We will add these results to Table 2 in the revision.