acceptodds
Under review as a conference paper at ICLR 2027

Phy-Judger: Learned Physics Rewards for Joint Prompt and Video RL

Abstract

Video generators produce visually convincing clips that still violate basic physics, such as a hand passing through a tabletop. Prior reinforcement learning (RL) methods for physical video generation reward the generator with hand-designed verifiers, each checking one property from object tracks, 3D estimates, or simulation states. These measurements are unreliable in open-world videos, and violations of other properties go unpenalized. We instead learn the reward from a physics preference dataset of 10k prompt–video pairs. The prompts cover 25 sub-scenes in five categories: human motion, indoor and everyday objects, industry and robotics, traffic and vehicles, and nature and materials. The label of each pair is known from how the pair is built, e.g., a real clip over a generated one, so annotators only write rationales. A rationale scores each video on six physical dimensions, such as mechanics and geometry, and lists each violation with its time interval and bounding box. On these data we train Phy-Judger, a VLM reward model that writes a rationale before scoring a video, so each score is tied to specific violations. We also train a prompt policy jointly with the video policy; it rewrites each prompt to add missing physical conditions, such as material, contact, and support. Joint RL with Phy-Judger raises the VBench-2.0 Physics score of MiniMax-H3 from 75.80 to 78.83 and its PhyWorldBench physical commonsense score from 0.155 to 0.203. We will release the code, checkpoints and datasets.

Primary area:
generative models
Keywords:
Physical video generation

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_m3riOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper learns a physics reward for video-generator post-training. It constructs 10,114 preference pairs across 25 sub-scenes, using construction-based labels and rationales that identify and localize physical violations. These annotations train Phy-Judger, a VLM that explains its judgments and scores six physical dimensions. The judge, together with auxiliary quality rewards, guides joint RL of a prompt rewriter and MiniMax-H3. Validation includes held-out judge evaluation, component ablations, two physics benchmarks, and blinded human comparisons.

Critical strengths

The gains extend beyond the optimized judge. VBench Physics rises from 75.80 to 78.83, and PhyWorldBench physical commonsense rises from 0.155 to 0.203 (Tables 4–5, pp. 8–9). Human judgments also favor joint RL over the base model, 49.1% versus 33.1%, with randomized presentation, hidden method names, and matched seeds (Section 5.1, p. 6; Table 6, p. 9). Together, these results support an improvement in visible physical plausibility rather than merely a higher learned-reward score.

The annotation study offers useful practical evidence. In the 2,000-pair ablation, adding few-shot exemplars reduces dimension MAE from 1.42 to 0.46 (Table 1, p. 6). Appendix A.4 also groups shared prompts, videos, source clips, and near-duplicates within splits. These choices make the study of annotation quality more informative and the held-out evaluation more credible.

Critical weaknesses

The visual-grounding claim needs direct validation. Teachers receive the construction label and must make score orderings consistent with it; the label-consistency checker then reads only the rationale, not the videos (Appendix A.3, pp. 15–16). The UNSUPPORTED option is a meaningful safeguard, but textual consistency does not establish that a cited violation, timestamp, or box is correct. Tables 1–2 evaluate preferences and scores, not the accuracy of this localized evidence. This leaves a central claim—scores grounded in correctly identified visual violations—less well supported than the end-to-end scoring results. It is an evidence gap, not proof that the rationales are wrong.

The added value of preference-training the reward is not isolated. Tables 1–2 compare fine-tuned judges, and Table 3 compares policy-training variants, but none compares RL using Phy-Judger against its untuned Qwen3.5-9B backbone with the same rubric. The existing ablations therefore do not establish how much the new supervision improves downstream RL beyond pretrained VLM capabilities. Reward accounting also matters here: Appendix A.7 assigns a total weight of 0.3 to HPS/VideoAlign rewards, although Section 5.1 introduces these primarily as quality-tracking measures. Their gains should not be interpreted as a by-product of physical optimization alone. The independent physics improvements remain meaningful, but currently support the full recipe more strongly than the specific benefit of learning Phy-Judger.

Questions for authors

Clarification and targeted validation: Were the 200 held-out human preferences and dimension scores collected without revealing construction labels? A small label-blind audit of retained rationales, checking whether their violations and localizations match the videos, would directly address the grounding concern.

Clarification: Does Appendix A.7's 0.7/0.3 reward split apply to every RL variant in Table 3 and to both policies? Please state the complete objective and identify which evaluation metrics are also optimized.

Decision-relevant comparison: How does joint RL using untuned Qwen3.5-9B compare with Phy-Judger under identical rubrics, frame sampling, rollout budgets, and auxiliary rewards? A controlled advantage on the independent physics evaluations would establish the downstream value of the preference supervision.

6 comments

We sincerely thank the reviewer for the detailed and constructive feedback. Below we address each of the raised concerns:

W1: Visual-grounding claim needs direct validation.
  • The visual-grounding claim needs direct validation. Teachers receive the construction label and must make score orderings consistent with it; the label-consistency checker then reads only the rationale, not the videos (Appendix A.3, pp. 15–16). The UNSUPPORTED option is a meaningful safeguard, but textual consistency does not establish that a cited violation, timestamp, or box is correct. Tables 1–2 evaluate preferences and scores, not the accuracy of this localized evidence. This leaves a central claim—scores grounded in correctly identified visual violations—less well supported than the end-to-end scoring results. It is an evidence gap, not proof that the rationales are wrong.

Thank you for this insightful suggestion. We directly validate the cited violations, timestamps and boxes in the rationales.

We sample 300 video pairs with their rationales, covering all three pair types (150 real–synthetic, 69 strong–weak and 81 correct–perturbed) and all annotation tiers. Three human experts, Gemini-4-Argon and Opus-5.5 first see only the videos, without the labels or rationales, and mark the time interval and bboxes of each physical violation (stage 1). They then see the rationale and check whether each cited violation is visible (stage 2).

For each annotator, we report the ratio of cited violations that are visible. tIoU / IoU measure the temporal / bounding-box overlap between each cited violation confirmed visible and the same violation marked by the annotator before seeing the rationale (stage 1). As an upper bound, we compute the same metrics between every two human experts and average over the three pairs.

Visible violation acc. ↑tIoU (time) ↑IoU (bbox) ↑
Human 189.3%0.570.50
Human 292.3%0.590.52
Human 392.3%0.590.51
Gemini-4-Argon87.7%0.500.45
Opus-5.590.0%0.520.48
Inter-human (upper bound)–0.650.60

87.7%–92.3% of the cited violations are visible, and the timestamps and boxes match human experts (tIoU 0.57–0.59, IoU 0.50–0.52) close to the inter-human agreement (0.65 / 0.60).

We also check the rationales generated by the trained Phy-Judger itself on the 200 held-out test pairs with the same protocol.

Visible violation acc. ↑tIoU (time) ↑IoU (bbox) ↑
Phy-Judger71.5%0.480.44
Inter-human (upper bound)–0.670.62

The training rationales are thus well grounded. The rationales of Phy-Judger are less precise but still largely grounded: 71.5% of the cited violations are visible, and its localization reaches over 70% of the inter-human upper bound, far above the untuned Qwen3.5-9B backbone evaluated with the same protocol (tIoU 0.23, IoU 0.20). We will add these results to Table 2 in the revision.

W2: RL results with untuned Qwen3.5-9B backbone with the same rubric. And single Phy-Judger reward.
  • The added value of preference-training the reward is not isolated. Tables 1–2 compare fine-tuned judges, and Table 3 compares policy-training variants, but none compares RL using Phy-Judger against its untuned Qwen3.5-9B backbone with the same rubric. The existing ablations therefore do not establish how much the new supervision improves downstream RL beyond pretrained VLM capabilities. Reward accounting also matters here: Appendix A.7 assigns a total weight of 0.3 to HPS/VideoAlign rewards, although Section 5.1 introduces these primarily as quality-tracking measures. Their gains should not be interpreted as a by-product of physical optimization alone. The independent physics improvements remain meaningful, but currently support the full recipe more strongly than the specific benefit of learning Phy-Judger.

Thank you for this valuable suggestion. We add two RL variants under the same setting: (i) untuned Qwen3.5-9B + auxiliary rewards, which replaces Phy-Judger with its untuned backbone; and (ii) Phy-Judger only, without auxiliary rewards.

Phy-Judger ↑VB-P ↑VB-C ↑MQ ↑TA ↑HPS-G ↑HPS-P ↑
MiniMax-H32.9875.8075.231.572.142.946.76
(i) Untuned Qwen3.5-9B + auxiliary3.0075.9275.401.782.673.307.32
(ii) Phy-Judger only3.7078.9078.531.502.252.606.81
Phy-Judger + auxiliary (Ours)3.6278.8378.041.812.733.297.52

(i) The untuned backbone barely improve physics (+0.12 VB-P), since the untuned backbone is an unreliable reward: only 77.5% of its outputs are valid JSON, and its preference accuracy is 43.5% vs. 72.5% for Phy-Judger in the reward model test set. Under the same recipe, Phy-Judger adds +2.91 VB-P and +2.64 VB-C over it.

(ii) With Phy-Judger alone, RL achieves the full physics gain (+3.03 VB-P, +3.30 VB-C) without any auxiliary reward. Thus, essentially all of the physics gains come from the learned Phy-Judger. The 0.3 auxiliary rewards act as a standard RL regularizer against reward hacking: without them, the Phy-Judger score is the highest (3.70), but MQ and HPS-G drop below the base model.

We will revise Section 5.1 to clearly state that MQ, TA and HPS serve both as auxiliary rewards (total weight 0.3) and as general-quality metrics.

Q1: Clarification about the 200 held-out test pairs.
  • Clarification and targeted validation: Were the 200 held-out human preferences and dimension scores collected without revealing construction labels? A small label-blind audit of retained rationales, checking whether their violations and localizations match the videos, would directly address the grounding concern.

Thank you for this helpful question. Yes. The 200 held-out pairs are the test set of the reward model. Their annotators see only the original prompt and the two videos, and give their own preference, per-dimension scores and rationale under the same rubric, without the construction labels.

The label-blind audit of the retained training rationales is reported in our response to W1. We further audit these held-out rationales in the same way: three new annotators first mark the time interval and bbox of each violation, then check whether each violation cited in the rationale is visible; tIoU / IoU are computed between their first-stage annotations and the cited violations confirmed visible. The agreement between every two new annotators serves as the upper bound.

ComparisonVisible violation acc. ↑tIoU (time) ↑IoU (bbox) ↑
Held-out rationale vs. new annotators92.5%0.670.61
Inter-annotator (upper bound)–0.670.62

92.5% of the cited violations are visible, and the localization nearly reaches the upper bound. Thus, the held-out annotations are reliable and grounded in visible violations.

Q2: Clarification about the reward setting.
  • Clarification: Does Appendix A.7's 0.7/0.3 reward split apply to every RL variant in Table 3 and to both policies? Please state the complete objective and identify which evaluation metrics are also optimized.

Thank you for this careful question. Yes, the same 0.7/0.3 reward split is used for every RL variant in Table 3 and for both the video policy and the prompt policy. The complete objective is

max⁡θ,ψJvideo(θ)+Jprompt(ψ),Au=∑d0.76⋅rdu−μdsd+δ+∑g0.34⋅rgu−μgsg+δ,\max_{\theta,\psi}\\ \mathcal{J}_{\text{video}}(\theta)+\mathcal{J}_{\text{prompt}}(\psi),\qquad A^u=\sum_{d}\frac{0.7}{6}\cdot\frac{r^u_d-\mu_d}{s_d+\delta}+\sum_{g}\frac{0.3}{4}\cdot\frac{r^u_g-\mu_g}{s_g+\delta},

where dd runs over the six Phy-Judger dimensions and gg over HPS-G, HPS-P, MQ and TA. Jvideo\mathcal{J}_{\text{video}} is the Flow-GRPO objective (Eq. 2) with advantage Aj,kA^{j,k}, where μ\mu averages over the NN videos of the same rewrite; Jprompt\mathcal{J}_{\text{prompt}} is its token-level GRPO counterpart with advantage AjA^{j}, where μ\mu averages over the MM rewrites of the same prompt.

Among the metrics, Phy-Judger, MQ, TA, HPS-G and HPS-P are optimized, while VB-P, VB-C, PhyWorldBench and the human evaluation are not optimized and serve as independent evaluators. We will state this in Section 4.3 of the revision.

Q3: RL results with untuned Qwen3.5-9B backbone under identical settings.
  • Decision-relevant comparison: How does joint RL using untuned Qwen3.5-9B compare with Phy-Judger under identical rubrics, frame sampling, rollout budgets, and auxiliary rewards? A controlled advantage on the independent physics evaluations would establish the downstream value of the preference supervision.

Thank you for this insightful question. Yes. Following your suggestion, we compare joint RL with Phy-Judger vs. its untuned Qwen3.5-9B backbone under identical settings: same rubric, frame sampling, rollout budget, and auxiliary rewards. The untuned run barely improves physics (+0.12 VB-P, +0.17 VB-C), while Phy-Judger adds +2.91 VB-P and +2.64 VB-C.

The detailed results are shown in W2. We hope this controlled comparison helps establish the value of the preference supervision.

Reviewer_vNxjOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper builds video preference pairs using real and generated clips, different generators, and physically altered prompts. Teachers explain the assigned preferences with localized violations and dimension scores. These annotations train Phy-Judger, which supplies rewards for jointly training a video generator and a prompt rewriter. The full objective also includes general quality rewards. The paper reports improved physical plausibility on external benchmarks and in a blinded human comparison.

Critical strengths
  1. A realistic-looking video can still contain broken contact, motion, or object behavior. Feedback that identifies the actual violation could be more useful than a single general preference score.
  2. I appreciate the rejection of unsupported preferences, source-aware train-test splitting, and consistent annotation format. These are meaningful steps beyond simply assuming that every constructed pair is correct.
Critical weaknesses
  1. The labels can still reward source recognition. Real clips and outputs from stronger generators are assigned as winners before annotation. Removing unsupported pairs helps, but does not establish that the retained labels reflect physics rather than source quality. I'd like to see a label-blind audit and results on pairs from the same generator.
  2. The annotation prompt forces agreement where tradeoffs may exist, which requires every dimension's score ordering to agree with the overall winner. It can corrupt the very dimension-specific rewards the method relies on.
  3. The reward-model ablations compare trained variants, but not the untuned backbone using the same rubric. Preference accuracy also does not establish that the reported boxes and timestamps identify real violations. I cannot tell how much physical judgment the training actually adds.
  4. The prompt-only baseline is too weak for the main claim. It uses an untrained rewriter, whereas the proposed system learns to rewrite prompts. Generator-only RL helps, but does not show what joint training adds over a trained rewriter with a frozen generator. I'd suggest using PhyPrompt (https://arxiv.org/abs/2603.03505) as a baseline.
Questions for authors

See Weaknesses.

3 comments

We sincerely thank the reviewer for the thoughtful feedback. Below we address each of the raised concerns:

W1: Label-blind audit and results on pairs from the same generator.
  • The labels can still reward source recognition. Real clips and outputs from stronger generators are assigned as winners before annotation. Removing unsupported pairs helps, but does not establish that the retained labels reflect physics rather than source quality. I'd like to see a label-blind audit and results on pairs from the same generator.

Thank you for this insightful suggestion. We agree that same-generator pairs provide a direct test of whether the labels reward source recognition. Our correct–perturbed pairs (27% of the data) are such pairs: both videos come from the same generator and seed and differ only in the prompt. We will state this explicitly in the main text.

Label-blind audit. Following the reviewer's suggestion, we sample 300 retained pairs from our dataset, stratified by pair type in the dataset proportions (150 real–synthetic, 69 strong–weak and 81 correct–perturbed pairs). Three human experts, Gemini-4-Argon and Opus-5.5 annotate the preference and per-dimension scores based only on the videos and the physics rubric, without seeing the labels or rationales. We report their agreement with the retained labels on all pairs, on cross-source pairs (real–synthetic and strong–weak), where source quality could serve as a cue, and on same-generator pairs (correct–perturbed), where it cannot.

Human 1Human 2Human 3Gemini-4-ArgonOpus-5.5
All pairs (300)90.7%94.7%94.0%88.0%91.7%
Cross-source pairs (219, real–syn. & strong–weak)90.9%95.0%93.6%87.7%92.2%
Same-generator pairs (81, correct–perturbed)90.1%93.8%95.1%88.9%90.1%

If the retained labels rewarded source recognition rather than physics, annotators who judge only physics would disagree with them more often on cross-source pairs than on same-generator pairs. This is not the case: the agreement is 87.7%–95.0% on cross-source pairs and 88.9%–95.1% on same-generator pairs; for each annotator, the two differ by at most 2.1 points, with no consistent direction. This indicates that the retained labels reflect physical plausibility rather than source quality.

Results on same-generator pairs. Phy-Judger trained on these labels does not rely on source cues either. On the 54 same-generator (correct–perturbed) pairs among the 200 held-out test pairs, Phy-Judger reaches 72.2% preference accuracy and 0.42 dimension MAE, on par with 72.5% and 0.42 on all 200 pairs (Table 1).

W2: Tradeoffs in dimension-specific rewards.
  • The annotation prompt forces agreement where tradeoffs may exist, which requires every dimension's score ordering to agree with the overall winner. It can corrupt the very dimension-specific rewards the method relies on.

Thank you for this insightful observation. We address this with two experiments.

Real tradeoffs are rare. On the 300 label-blind audited pairs described in our response to W1, where human experts give per-dimension scores, only 7.7% of the pairs have a dimension whose ordering is opposite to the overall preference (ties and non-applicable dimensions do not count). This follows from how the pairs are built:

  • Correct–perturbed (27%): both videos share the generator, seed, entities, setting and camera, and the perturbed prompt adds exactly one violation in one dimension (Appendix A.2). The two videos thus differ mainly in that dimension and tie in the others.
  • Real–synthetic (50%): the real clip is physically correct in every dimension, so the generated video rarely scores higher in any of them.
  • Strong–weak (23%): the failures of a weaker generator tend to co-occur across dimensions; e.g., when an object deforms or vanishes (Geometry, Instance), its motion and contacts (Motion, Mechanics) also break.
  • Pairs in which neither video is clearly better, where tradeoffs are most likely, are marked UNSUPPORTED and removed.

Forcing agreement can therefore affect the dimension scores of at most these 7.7% of pairs.

Allowing tradeoffs does not help. We re-annotate the same 2,000 training pairs of Table 1 with a prompt that allows tradeoffs, and retrain Phy-Judger. Both variants use the same 2,000 pairs and the same training setting; the forced-agreement variant is exactly the last row of Table 1. Both are tested on the 200 held-out pairs. Dimension acc. is the accuracy of the per-dimension score ordering.

Preference label acc. ↑Dimension MAE ↓Dimension acc. ↑
Forced agreement (ours, Table 1)72.50.4264.7
Tradeoffs allowed71.50.4265.0

Allowing tradeoffs raises dimension accuracy by only 0.3 points, while preference accuracy drops. Thus forcing agreement does not hurt the dimension-specific rewards, consistent with real tradeoffs being rare. We will add this to the revision.

W3: Ablation on the untuned backbone, and boxes and timestamps.
  • The reward-model ablations compare trained variants, but not the untuned backbone using the same rubric. Preference accuracy also does not establish that the reported boxes and timestamps identify real violations. I cannot tell how much physical judgment the training actually adds.

Thank you for this valuable suggestion. We evaluate the untuned Qwen3.5-9B backbone with the same rubric as Phy-Judger on the 200 held-out human-annotated test pairs of Tables 1–2. Three human experts also mark the time interval and bounding box of each violation in these pairs.

Both models are scored against the human annotations. tIoU / IoU measure the temporal / bounding-box overlap between each cited violation confirmed visible and the same violation marked by the human experts without seeing the model output, averaged over the three experts. Invalid outputs of the untuned backbone count as wrong preferences with 0 tIoU / IoU.

To separate physical judgment from format following, we also report each metric only on the 155 pairs (77.5%) where the untuned backbone output is valid JSON (in parentheses), with Phy-Judger evaluated on the same 155 pairs. The upper bound (last row) is the tIoU / IoU between every two human experts, averaged over the three expert pairs.

Valid JSON format ↑Preference label acc. ↑tIoU (time) ↑IoU (bbox) ↑
Qwen3.5-9B (untuned)77.5 %43.5 % (56.1 %)0.23 (0.30)0.20 (0.26)
Phy-Judger100 %72.5 % (73.5 %)0.48 (0.49)0.44 (0.44)
Inter-human (upper bound)––0.670.62

Even on its valid outputs, the untuned backbone reaches only 56.1% preference accuracy, while Phy-Judger reaches 73.5% on the same 155 pairs; its localization (tIoU 0.30, IoU 0.26) also remains far below that of Phy-Judger (0.49 / 0.44). Thus the gain does not come from learning the output format: on the pairs where the backbone already follows the format, training still improves preference accuracy, tIoU and IoU. This shows that the training adds substantial physical judgment beyond the backbone. We will add this to the revision.

W4: Using PhyPrompt as a rewriter baseline.
  • The prompt-only baseline is too weak for the main claim. It uses an untrained rewriter, whereas the proposed system learns to rewrite prompts. Generator-only RL helps, but does not show what joint training adds over a trained rewriter with a frozen generator. I'd suggest using PhyPrompt (https://arxiv.org/abs/2603.03505) as a baseline.

Thank you for this constructive suggestion. We add PhyPrompt, an RL-trained prompt rewriter, as a baseline. It rewrites the prompts for the frozen MiniMax-H3. We compare it with generator-only RL and with our joint training.

RewriterGeneratorPhy-Judger ↑VB-P ↑VB-C ↑
MiniMax-H3–frozen2.9875.8075.23
+ Untrained rewriter (prompt-only)untrainedfrozen3.1276.4075.59
+ PhyPrompttrainedfrozen3.3877.1576.73
Generator-only RL (Table 3)–trained3.4477.2076.92
Joint training (Ours)trainedtrained3.6278.8378.04

PhyPrompt is stronger than the untrained rewriter and on par with generator-only RL. Joint training outperforms both, by +1.68 VB-P and +1.31 VB-C over PhyPrompt, and by +1.63 VB-P and +1.12 VB-C over generator-only RL. Training both policies on the same rollouts lets the rewriter learn which physical conditions this generator needs stated, and the generator learn to render them. We will add PhyPrompt as a baseline in the revision.

Reviewer_X1LrOpenReview ↗
Rating
3
Confidence
3(+1)
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper learns a physical reward model, Phy-Judger, from roughly 10k video preference pairs annotated with explanations and localized physical violations. The reward is then used to jointly optimize a prompt rewriter and a video generator through reinforcement learning. Experiments on MiniMax-H3 show improvements on VBench-2.0 and PhyWorldBench, supported by ablations and human evaluation.

Critical strengths
  1. Learning a physical reward from diverse video comparisons is a useful approach to covering errors beyond those captured by specialized physical verifiers. The localized explanations also make the judge’s decisions easier to inspect.
  2. The evaluation goes beyond the reward being optimized: independent benchmarks and human comparisons both favor the resulting videos. This strengthens the evidence that the method improves visible physical plausibility.
Critical weaknesses
  1. Preference labels are assigned by construction, and annotators see these labels when writing explanations. Although unsupported pairs are filtered out, recovering a label from its explanation checks consistency rather than whether the explanation is visually correct. This leaves some uncertainty about bias in the reward supervision.
  2. Generator optimization is evaluated only on MiniMax-H3. The results support the method on this model, but its usefulness across generators with different physical failure patterns remains untested.
Questions for authors
  1. The joint training uses four prompt rewrites and four videos per rewrite. With the total sampling budget fixed, how sensitive is the method to this allocation? This would help clarify the practical trade-off between exploring different prompts and sampling more videos from each prompt.
  2. Table 2 shows that localization improves judge accuracy. Does this also lead to better videos after RL? A comparison using rewards trained with and without localization would help establish whether this component benefits the full framework.
4 comments

We sincerely thank the reviewer for the thorough review and constructive feedback. Below we address each of the raised concerns:

W1: Visual correctness of the dataset explanations and potential bias in reward supervision.

Preference labels are assigned by construction, and annotators see these labels when writing explanations. Although unsupported pairs are filtered out, recovering a label from its explanation checks consistency rather than whether the explanation is visually correct. This leaves some uncertainty about bias in the reward supervision.

Thanks for this insightful comment. We test both concerns with a label-blind re-annotation. We sample 300 video pairs, together with their explanations and labels, from our dataset, stratified by pair type in the dataset proportions (150 real–synthetic, 69 strong–weak and 81 correct–perturbed pairs). Each pair is re-annotated by three human experts, Gemini-4-Argon and Opus-5.5.

The new annotators first see only the videos, without the labels or explanations. They are asked to: (a) give their own preference label and scores for the two videos; (b) mark time interval and bounding boxes of each physical violation. Then they see the explanation and the label, and (c) check whether each violation in the explanation is visible in the video.

The table below reports, for each annotator, the agreement with the original label and the ratio of violations in the explanation that are visible. tIoU / IoU measure the temporal / bounding-box overlap between each violation confirmed visible and the same violation marked by the annotator.

Label agreement ↑Visible violation acc. ↑tIoU (time) ↑IoU (bbox) ↑
Human 190.7 %89.3 %0.570.50
Human 294.7 %92.3 %0.590.52
Human 394.0 %92.3 %0.590.51
Gemini-4-Argon88.0 %87.7 %0.500.45
Opus-5.591.7 %90.0 %0.520.48
Inter-human (upper bound)––0.650.60

Visual correctness. The annotators confirm that 87.7%–92.3% of the violations in the explanations are visible in the videos. Since violation boundaries are inherently ambiguous, we use the average agreement between every two human experts as the upper bound for localization (last row). Against the human experts, the explanations reach 83%–91% of this upper bound.

Bias in the labels. One may worry that the construction labels encode a prior rather than physics: a real clip may win by image quality, and a stronger generator is not always physically better. We therefore also report the agreement on the 150 pairs of two generated videos. It stays close to the agreement on all pairs in the table above: 86.7%–94.7% on these 150 pairs, within 1.3 points of all pairs for every annotator. Within them, it is 84.1%–95.7% on the 69 strong–weak pairs and 88.9%–95.1% on the 81 correct–perturbed pairs.

This indicates that the explanations are visually correct, not only consistent with the labels, and that the reward is supervised by labels that reflect physical plausibility rather than image quality or generator strength.

W2: Experiments across generators with different physical failure patterns.
  • Generator optimization is evaluated only on MiniMax-H3. The results support the method on this model, but its usefulness across generators with different physical failure patterns remains untested.

Thanks for valuable suggestion. We apply joint RL with Phy-Judger to Wan2.1-1.3B, which has a different and much more frequent physical failure pattern (objects deform or drift across views, and motions are less plausible). The results are shown in the table below.

Phy-Judger ↑\uparrowVB-P ↑\uparrowVB-C ↑\uparrowMQ ↑\uparrowTA ↑\uparrowHPS-G ↑\uparrowHPS-P ↑\uparrow
Wan2.1-1.3B2.3459.7664.250.811.952.576.02
Phy-Wan2.1-1.3B3.1764.1469.761.202.412.896.66

After joint RL, Wan2.1-1.3B improves by 4.38 on VB-P and 5.51 on VB-C. This shows that our method remains effective on generators with different physical failure patterns. We will add this to the revision.

Q1: Different prompts and sampling more videos from each prompt.

  • The joint training uses four prompt rewrites and four videos per rewrite. With the total sampling budget fixed, how sensitive is the method to this allocation? This would help clarify the practical trade-off between exploring different prompts and sampling more videos from each prompt.

Thank you for this insightful question. We fix the total budget at G = M × N = 16 videos per user prompt and vary the allocation between the number of rewrites M and the number of videos per rewrite N. All other settings are kept the same.

M (rewrites per prompt)N (videos per rewrite)Phy-Judger ↑VB-P ↑VB-C ↑
MiniMax-H3 (no RL)–2.9875.8075.23
1163.4277.1076.70
283.5678.5877.76
4 (Ours)43.6278.8378.04
823.5478.4977.87
1613.3676.9276.50

The method is not sensitive to the allocation as long as both M and N are at least 2, and the balanced M = N = 4 performs best. Performance drops only at the two extremes (M=1 or N=1). Thus, both exploring different prompts and sampling multiple videos from each prompt matter, and a balanced allocation is a good default in practice.

Q2: Localization leads to better videos after RL.

  • Table 2 shows that localization improves judge accuracy. Does this also lead to better videos after RL? A comparison using rewards trained with and without localization would help establish whether this component benefits the full framework.

Thank you for this thoughtful question. Yes. We train a reward without localization by removing all time intervals and bounding boxes from the training data, and then run joint RL. The results are shown in the table below.

Phy-Judger ↑\uparrowVB-P ↑\uparrowVB-C ↑\uparrow
MiniMax-H32.9875.8075.23
W/o localization3.4277.4577.78
W/ localization3.6278.8378.04

The reward with localization leads to better videos after RL, improving VB-P by 1.38 (77.45 → 78.83) and VB-C by 0.26 (77.78 → 78.04) over the reward without localization.

I appreciate the author's comprehensive response and additional experiments. My concerns have been largely addressed, and the identified weaknesses have been adequately responded to. I will maintain my rating and increase confidence to 3.

Dear Reviewer X1Lr,

Thank you very much for your follow-up and for carefully reviewing our response and additional experiments. We are glad that our clarifications have largely addressed your concerns, and we appreciate your increased confidence. Your constructive feedback has helped improve our paper.

Thank you again for your time, effort, and support.

Best regards,

Authors of Submission 45142

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.