- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies whether multimodal models can reliably evaluate AI-generated embodied-manipulation videos. The authors introduce RoboJudge-Bench, containing 800 videos rated for physical adherence and instruction alignment, together with a training set of 12,351 videos. They adapt Qwen3.5-9B through SFT followed by GRPO using an exact-score matching reward. The resulting RoboJudge-9B improves agreement with human ratings on the proposed benchmark and shows improvements over the base model on two external benchmarks.
Critical strengths
The evaluation task is practically relevant. Separating physical adherence from instruction alignment distinguishes physically implausible task completion from plausible behavior that fails to achieve the goal. The data cover multiple embodiments, source corpora, and video generators. Splitting training and test data by source clip is an appropriate precaution against overlap between related generated videos. The study includes extensive baselines, SFT/RL ablations, external benchmarks, and source-clip cluster-bootstrap analysis, providing useful evidence for improved overall agreement with human ratings.
Critical weaknesses
The methodological increment is limited, and the independent value of RL is not sufficiently established. The training recipe combines an existing model with standard SFT and GRPO using an exact-score matching reward, without introducing a new architecture or optimization mechanism. In Table 3, SFT accounts for most of the overall correlation improvement, from 0.401 to 0.700, while GRPO increases it to 0.719. However, IA MSE worsens from 0.671 to 0.688, and the Physion-Eval metric decreases from 0.105 to 0.093 in Table 4. Without a continued-SFT baseline with a matched training budget, it remains unclear whether RL is more effective than additional supervised training.
Questions for authors
The method uses standard SFT and GRPO, with an RL reward based on exact agreement between predicted and human scores. Beyond applying this existing training pipeline to embodied-video scoring, what is the methodological contribution? Do the experiments establish a need for task-specific optimization or provide a generalizable algorithmic insight beyond the finding that domain-specific fine-tuning improves scoring agreement? GRPO uses a binary exact-match reward, assigning zero to both one-point and four-point errors and ignoring the ordinal structure of the scores. Is this objective misaligned with the primary evaluation metrics, Pearson correlation and MSE? After RL, IA MSE worsens from 0.671 to 0.688, and the Physion-Eval metric decreases from 0.105 to 0.093. How do the authors explain these regressions: does optimizing discrete label agreement fail to improve error magnitude or generalization? Have distance-aware rewards been compared to test this explanation?
4 comments
Reply to Reviewer 7Vps
Dear Reviewer 7Vps,
As the discussion period comes to a close, we would like to restate two points that bear directly on your concerns.
-
Contribution: we do not claim a new training algorithm. The contribution is the evaluation problem itself — the benchmark, the training set, and the finding that general MLLMs are unreliable on physical adherence while specialized video judges do not transfer. We will state this positioning explicitly in the revision.
-
RL validity: with the same checkpoint, the same 1,600 samples, and 200 training steps, SFT reaches 0.700, continued SFT 0.703, and RL 0.719, with the gain concentrated in Physical Adherence. The IA MSE and Physion-Eval changes have 95% CIs that include zero, while the PA gains are significant. We will report SFT and SFT+RL side by side and scope the RL claim to PA.
If the concern is novelty framing, we will fix it; if it is the validity of the RL claim, we hope this control addresses it. We would be grateful for your reconsideration before the decision.
Gentle Follow-up on Our Rebuttal
Thank you again for your constructive feedback and helpful suggestions.
We would like to kindly follow up on our rebuttal responses. We have conducted additional experiments and analyses to address your concerns, and would greatly appreciate it if you could take a look when you have a chance.
Please let us know if you have any remaining questions or concerns. We would be happy to provide further clarification.
Thank you for your time and consideration!
Your explanations address most of my concerns. However, I still do not see sufficient methodological novelty, so I will maintain my original score.
Reviewer Response to Reviewer 7Vps
We thank the reviewer for the careful evaluation and constructive feedback. We address each concern below.
Weakness 1: Methodological Novelty and the Role of RL
Our primary contribution lies in evaluation: the first benchmark for embodied video generation judges, the first large-scale training dataset, and a state-of-the-art open-source judge, to the best of our knowledge. SFT improves overall correlation from 0.401 to 0.700, while RL further improves it to 0.719. We position RL as an effective refinement on top of SFT, rather than our main methodological contribution.
Weakness 2: Does RL outperform additional SFT?
Yes. We conduct a controlled, data-matched comparison, starting from the same SFT checkpoint. Continued SFT uses exactly the same 1,600 samples as RL, with both trained for 200 steps.
Under the same data budget, continued SFT provides minimal improvement, while RL yields additional gains, including a significant improvement in PA correlation (+0.047, 95% CI [+0.010, +0.085]).
Weakness 3: Exact-Match vs. Distance-Aware Rewards
We compare exact-match reward against two distance-aware alternatives: tiered (1/0.5/0) and linear (1 − |d|/4). All models start from the same SFT checkpoint and train for 200 steps.
Although correlations are comparable, distance-aware rewards compress predictions toward the middle of the rating scale, reducing score variance and nearly eliminating perfect scores across checkpoints. Partial credit for near-correct predictions may encourage conservative, middle-range scores. Exact-match reward better preserves score discrimination, motivating our design choice.
Weakness 4: Why do IA MSE and Physion-Eval performance decrease after RL?
We report paired bootstrap 95% confidence intervals for the changes from SFT to SFT+RL:
RL significantly improves PA correlation and MSE, while changes in IA MSE and Physion-Eval are not statistically significant (CIs include zero). For Physion-Eval, the SFT baseline is the SFT model reported in the paper.
Both OOD benchmarks differ substantially from our training distribution: VideoPhy2 contains no embodied data, while Physion-Eval uses a different binary classification task and prompting format. RL does not necessarily yield consistent improvements under such distribution shifts, as also discussed in Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model? (NeurIPS 2025) and Can GRPO Help LLMs Transcend Their Pretraining Origin?
Importantly, our training pipeline substantially improves OOD performance over the base model:
Overall, RoboJudge outperforms the base model across all OOD metrics, including the substantially different Physion-Eval benchmark. These results support the effectiveness of our training pipeline, while indicating that RL's clearest and statistically significant gains are in Physical Adherence.