acceptodds
Under review as a conference paper at ICLR 2027

Robojudge: Multimodal Language Models as Judges for Embodied Video Generation

Abstract

Video generation models are promising foundations for embodied world models, but only if their generated rollouts are physically valid and faithful to task instructions. Multimodal large language models (MLLMs) are increasingly used to judge these properties, yet their agreement with human judgments on embodied videos has not been systematically evaluated. To address this gap, we introduce ROBOJUDGE-BENCH, a benchmark for evaluating MLLMs as judges of embodied video generation. The benchmark measures two fundamental dimensions, Physical Adherence and Instruction Alignment, across 800 human-annotated videos from 8 source corpora and 10 video generators, spanning grippers, human hands, and dexterous hands. Experiments reveal uneven alignment between existing judges and human ratings: general-purpose MLLMs show lower agreement on physical adherence, while specialized video judges designed for general-domain physical reasoning do not consistently transfer to embodied scenarios. Motivated by these findings, we construct ROBOJUDGE-TRAIN, comprising 12K videos annotated with 25K human-verified scores and model-assisted rationales for Physical Adherence and Instruction Alignment. We fine-tune a 9B MLLM on this dataset with supervised learning followed by reinforcement learning, yielding ROBOJUDGE-9B. Among the model judges evaluated on ROBOJUDGE-BENCH, ROBOJUDGE-9B achieves the highest overall agreement, reaching a Pearson correlation of 0.719 with a statistically significant 0.089 gain over Gemini-3.7-Flash, the best-performing proprietary baseline in our evaluation. Training on ROBOJUDGE-TRAIN also yields gains on external benchmarks, including VideoPhy2 and Physion-Eval. We further use ROBOJUDGE-9B as a downstream evaluator to benchmark video generation models according to Physical Adherence and Instruction Alignment.

Primary area:
datasets and benchmarks
Keywords:
Embodied World Model,Video Quality Evaluation

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_7VpsOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies whether multimodal models can reliably evaluate AI-generated embodied-manipulation videos. The authors introduce RoboJudge-Bench, containing 800 videos rated for physical adherence and instruction alignment, together with a training set of 12,351 videos. They adapt Qwen3.5-9B through SFT followed by GRPO using an exact-score matching reward. The resulting RoboJudge-9B improves agreement with human ratings on the proposed benchmark and shows improvements over the base model on two external benchmarks.

Critical strengths

The evaluation task is practically relevant. Separating physical adherence from instruction alignment distinguishes physically implausible task completion from plausible behavior that fails to achieve the goal. The data cover multiple embodiments, source corpora, and video generators. Splitting training and test data by source clip is an appropriate precaution against overlap between related generated videos. The study includes extensive baselines, SFT/RL ablations, external benchmarks, and source-clip cluster-bootstrap analysis, providing useful evidence for improved overall agreement with human ratings.

Critical weaknesses

The methodological increment is limited, and the independent value of RL is not sufficiently established. The training recipe combines an existing model with standard SFT and GRPO using an exact-score matching reward, without introducing a new architecture or optimization mechanism. In Table 3, SFT accounts for most of the overall correlation improvement, from 0.401 to 0.700, while GRPO increases it to 0.719. However, IA MSE worsens from 0.671 to 0.688, and the Physion-Eval metric decreases from 0.105 to 0.093 in Table 4. Without a continued-SFT baseline with a matched training budget, it remains unclear whether RL is more effective than additional supervised training.

Questions for authors

The method uses standard SFT and GRPO, with an RL reward based on exact agreement between predicted and human scores. Beyond applying this existing training pipeline to embodied-video scoring, what is the methodological contribution? Do the experiments establish a need for task-specific optimization or provide a generalizable algorithmic insight beyond the finding that domain-specific fine-tuning improves scoring agreement? GRPO uses a binary exact-match reward, assigning zero to both one-point and four-point errors and ignoring the ordinal structure of the scores. Is this objective misaligned with the primary evaluation metrics, Pearson correlation and MSE? After RL, IA MSE worsens from 0.671 to 0.688, and the Physion-Eval metric decreases from 0.105 to 0.093. How do the authors explain these regressions: does optimizing discrete label agreement fail to improve error magnitude or generalization? Have distance-aware rewards been compared to test this explanation?

4 comments
AuthorsOpenReview ↗

Reviewer Response to Reviewer 7Vps

We thank the reviewer for the careful evaluation and constructive feedback. We address each concern below.

Weakness 1: Methodological Novelty and the Role of RL

Our primary contribution lies in evaluation: the first benchmark for embodied video generation judges, the first large-scale training dataset, and a state-of-the-art open-source judge, to the best of our knowledge. SFT improves overall correlation from 0.401 to 0.700, while RL further improves it to 0.719. We position RL as an effective refinement on top of SFT, rather than our main methodological contribution.

Weakness 2: Does RL outperform additional SFT?

Yes. We conduct a controlled, data-matched comparison, starting from the same SFT checkpoint. Continued SFT uses exactly the same 1,600 samples as RL, with both trained for 200 steps.

Model (Same Initialization, 200 Steps, 1,600 Samples)Overall r
SFT baseline0.700
Continued SFT (data-matched)0.703
RL (exact-match reward)0.719

Under the same data budget, continued SFT provides minimal improvement, while RL yields additional gains, including a significant improvement in PA correlation (+0.047, 95% CI [+0.010, +0.085]).

Weakness 3: Exact-Match vs. Distance-Aware Rewards

We compare exact-match reward against two distance-aware alternatives: tiered (1/0.5/0) and linear (1 − |d|/4). All models start from the same SFT checkpoint and train for 200 steps.

RewardOverall rPA Std. (Human: 1.16)Predicted PA=5 (Human: 91)
Exact (Ours)0.7190.8822
Tiered0.7150.746
Linear0.7180.720

Although correlations are comparable, distance-aware rewards compress predictions toward the middle of the rating scale, reducing score variance and nearly eliminating perfect scores across checkpoints. Partial credit for near-correct predictions may encourage conservative, middle-range scores. Exact-match reward better preserves score discrimination, motivating our design choice.

Weakness 4: Why do IA MSE and Physion-Eval performance decrease after RL?

We report paired bootstrap 95% confidence intervals for the changes from SFT to SFT+RL:

MetricSFTSFT + RLDifference [95% CI]
PA r0.6330.669+0.036 [+0.004, +0.068]
PA MSE0.8710.784−0.088 [−0.161, −0.012]
IA MSE0.6840.709+0.025 [−0.045, +0.100]
Physion-Eval J0.1050.093−0.012 [−0.061, +0.037]

RL significantly improves PA correlation and MSE, while changes in IA MSE and Physion-Eval are not statistically significant (CIs include zero). For Physion-Eval, the SFT baseline is the SFT model reported in the paper.

Both OOD benchmarks differ substantially from our training distribution: VideoPhy2 contains no embodied data, while Physion-Eval uses a different binary classification task and prompting format. RL does not necessarily yield consistent improvements under such distribution shifts, as also discussed in Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model? (NeurIPS 2025) and Can GRPO Help LLMs Transcend Their Pretraining Origin?

Importantly, our training pipeline substantially improves OOD performance over the base model:

ModelVideoPhy2 PC rVideoPhy2 SA rPhysion-Eval J
Qwen3.5-9B (Base)0.1790.4120.056
+ SFT0.2990.5520.105
+ SFT + RL (RoboJudge)0.3460.5720.093

Overall, RoboJudge outperforms the base model across all OOD metrics, including the substantially different Physion-Eval benchmark. These results support the effectiveness of our training pipeline, while indicating that RL's clearest and statistically significant gains are in Physical Adherence.

AuthorsOpenReview ↗

Reply to Reviewer 7Vps

Dear Reviewer 7Vps,

As the discussion period comes to a close, we would like to restate two points that bear directly on your concerns.

  • Contribution: we do not claim a new training algorithm. The contribution is the evaluation problem itself — the benchmark, the training set, and the finding that general MLLMs are unreliable on physical adherence while specialized video judges do not transfer. We will state this positioning explicitly in the revision.

  • RL validity: with the same checkpoint, the same 1,600 samples, and 200 training steps, SFT reaches 0.700, continued SFT 0.703, and RL 0.719, with the gain concentrated in Physical Adherence. The IA MSE and Physion-Eval changes have 95% CIs that include zero, while the PA gains are significant. We will report SFT and SFT+RL side by side and scope the RL claim to PA.

If the concern is novelty framing, we will fix it; if it is the validity of the RL claim, we hope this control addresses it. We would be grateful for your reconsideration before the decision.

AuthorsOpenReview ↗

Gentle Follow-up on Our Rebuttal

Thank you again for your constructive feedback and helpful suggestions.

We would like to kindly follow up on our rebuttal responses. We have conducted additional experiments and analyses to address your concerns, and would greatly appreciate it if you could take a look when you have a chance.

Please let us know if you have any remaining questions or concerns. We would be happy to provide further clarification.

Thank you for your time and consideration!

Your explanations address most of my concerns. However, I still do not see sufficient methodological novelty, so I will maintain my original score.

Reviewer_tC4jOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies whether multimodal large language models can reliably judge generated embodied videos. It introduces ROBOJUDGE-BENCH, which consists of 800 human-annotated videos from 8 source corpora and 10 generators, and measures two fundamental dimensions, Physical Adherence and Instruction Alignment. It also constructs ROBOJUDGE-TRAIN, comprising 12K videos annotated with 25K human-verified scores and model assisted rationales. Then, the authors fine-tune a 9B MLLM on ROBOJUDGE-TRAIN with supervised learning followed by reinforcement learning to produce ROBOJUDGE-9B. The authors compare 29 model-judge baselines on ROBOJUDGE-BENCH, including 10 closed-source MLLMs, 7 open-weight MLLMs, 7 specialized embodied MLLMs, and 5 specialized judge models.

Critical strengths
  1. The paper addresses a meaningful and practical problem. As MLLMs are increasingly used to evaluate generated videos, it is important to evaluate whether they can reliably judge embodied videos. Studying the reliability of these judges is useful for both comparing video generation models and selecting generated rollouts for downstream use.
  2. The paper presents a comprehensive study, covering the evaluation benchmark ROBOJUDGE-BENCH, the training dataset ROBOJUDGE-TRAIN, and the fine-tuned model ROBOJUDGE-9B. The authors go beyond identifying weaknesses in existing judges by providing data and a trained model to address them.
Critical weaknesses
  1. I have some concerns about the annotation protocol. For ROBOJUDGE-BENCH, each video is labeled by one annotator, cross-reviewed by another, and adjudicated by an expert. Why not ask multiple annotators to score each video independently and average their ratings? The training set is annotated with three closed-source MLLMs followed by human review, which may lead to bias between training and test sets.
  2. It is unclear whether higher Physical Adherence (PA) and Instruction Alignment (IA) scores translate into better downstream performance. A video may look physically plausible and follow the instruction while still containing subtle errors that matter for planning or policy learning. The paper evaluates agreement with human ratings, but does not test whether the judge can select rollouts that lead to better downstream task outcomes. Such an experiment would help establish the practical value of these scores.
Questions for authors

See weakness.

7 comments
AuthorsOpenReview ↗

Reviewer Response to Reviewer tC4j

We thank the reviewer for recognizing the significance of our problem and the completeness of our benchmark, training dataset, and judge model. We address each concern below.

Weakness 1: Why review and adjudication instead of averaging ratings?

This is a deliberate design choice, consistent with established benchmark practices. GPQA uses expert verification; TACRED Revisited employs third-party adjudication and reports much lower crowdworker–expert agreement (κ=0.55) than expert–expert agreement (κ=0.87). MMLU-Redux, ADE20K, and SWE-bench Verified also adopt review-based quality control. Most relevantly, VideoScore (EMNLP 2024) and VideoScore2 use trained annotators with review-based quality control rather than simple averaging.

Averaging may introduce annotation noise and compress scores toward the middle, obscuring distinctions between high- and low-quality videos. Our protocol instead requires rubric-grounded rationales, with disagreements resolved by experts, ensuring reliable, discriminative, and traceable annotations.

Weakness 2: Can the judge select rollouts that improve downstream policy performance?

Our primary contribution is evaluating embodied video generation judges, including the first benchmark and large-scale training dataset for this purpose, to our knowledge (see Global Comment). Instruction Alignment directly assesses task completion, while Physical Adherence evaluates physical correctness.

In world action models (WAMs), prior work primarily establishes that better video generation leads to stronger downstream policies:

  • DreamZero: A stronger video backbone improves action success (14B: 50% vs. 5B: 21%).
  • PhysisForcing: Improved physical consistency increases RoboTwin 2.0 success from 68.2% to 72.8%.
  • DreamGen: Video benchmark scores correlate positively with downstream policy success.

Consistently, our evaluation assigns PF-Wan14B a RoboJudge score of 3.77 vs. 2.58 for Wan2.2-TI2V-5B. WorldArena further shows that visual quality alone cannot capture task performance, motivating our focus on physical correctness and instruction completion.

We also validate RoboJudge through Best-of-N selection: on Wan2.2, selected videos improve over one-shot generation on 26/50 tasks, with only 4 degradations, outperforming GPT-5.5 in net gains (see Reviewer Z2vA).

We agree that evaluating judge quality through downstream robotic policy success is valuable and will discuss this direction as future work.

AuthorsOpenReview ↗

Reply to Reviewer tC4j

Dear Reviewer tC4j,

With the discussion period ending, we want to make sure our response to your two concerns is easy to find.

Annotation bias: benchmark labels are human-only (annotator + independent cross-review + expert adjudication). The three MLLMs were used only to draft rationales; all scores are human-assigned and verified. No model-generated score enters labels or training targets.

Downstream value: As discussed in our rebuttal, prior work already establishes that stronger video generation translates into better downstream policies — DreamZero (stronger video backbone improves action success), PhysisForcing (better physical consistency raises RoboTwin success), and DreamGen (video benchmark scores correlate with policy success). Our focus is therefore on evaluating video quality itself, not on retraining a policy. We further validate practical utility through Best-of-N selection with blind human GSB: RoboJudge-selected videos beat one-shot generation (26 better / 4 worse on Wan2.2 Bo5), with larger net gains than GPT-5.5 in 3 of 4 settings.

We hope this addresses both concerns, and would be grateful for your reconsideration before the decision.

AuthorsOpenReview ↗

Gentle Follow-up on Our Rebuttal

Thank you again for your constructive feedback and helpful suggestions.

We would like to kindly follow up on our rebuttal responses. We have conducted additional experiments and analyses to address your concerns, and would greatly appreciate it if you could take a look when you have a chance.

Please let us know if you have any remaining questions or concerns. We would be happy to provide further clarification.

Thank you for your time and consideration!

Reviewer_Z2vAOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces RoboJudge-Bench, a benchmark with 800 cases for evaluating multimodal judges of embodied video generation, with focus on physical adherence and Instruction alignment. The authors also construct a large scale training dataset and produce a RoboJudge-9B through SFT and GRPO. The resulting judge achieves stronger alignment with human ratings.

Critical strengths
  1. The work is solid and self-contained. The author built a benchmark, trained a VLM judge model, and listed the rankings of existing video generation models in the embodied tasks.

  2. The human annotation process of pre-calibration is reasonable and rigorous.

  3. The illustration of cases and failure modes is also clear.

Critical weaknesses
  1. The training set includes 831 real demonstrations, all with PA=5 and IA=5. However the benchmark contains only generated videos. Could the author explain the rationale of this?

  2. The paper proposed a benchmark, trained a highly aligned judge model, and tested SOTA video models and give the rankings. A promising extension is to use the judge to improve embodied video generation via RFT, GRPO, or Best-of-N selection. Given the limited rebuttal time window, a small BoN selection experiment with blinded human GSB comparisons would help demonstrate practical utility, while video generator training may be the future work.

  3. How does the training with rationales performance compare to training without it? A comparison of scoring performance and inference latency would help clarify the benefits and computational cost of rationale generation.

Questions for authors

see weaknesses

3 comments
AuthorsOpenReview ↗

Reviewer Response to Reviewer Z2vA

We thank the reviewer for recognizing the completeness of our work, annotation rigor, and failure mode analysis. We address each concern below.

Weakness 1: Why include real videos in training?

Our benchmark contains only generated videos, whereas training includes real demonstrations as gold-standard examples. Since generated videos deserving a perfect score of 5 are rare, collecting sufficient high-quality examples through generation and human annotation is costly and difficult to scale.

SFTOverall rPredicted PA=5 (Human: 91)
Without real videos0.70116
With real videos (Ours)0.70065

Adding real videos helps the judge learn to assign the highest score (16 → 65, closer to 91 human annotations) without sacrificing correlation.

Weakness 2: Best-of-N Selection

We thank the reviewer for this suggestion. We evaluate Wan2.2 (ID) and MiniMax-H3 (OOD, unseen during training and benchmarking) on 50 tasks each, generating five candidates per task. We compare RoboJudge and GPT-5.5 as Best-of-N selectors against one-shot generation, using blind Good/Same/Bad (GSB) comparisons voted on by five independent human evaluators.

SelectorWan2.2 Bo3Wan2.2 Bo5MiniMax-H3 Bo3MiniMax-H3 Bo5
GPT-5.517/30/321/25/49/39/213/31/6
RoboJudge21/27/226/20/410/36/415/29/6

Entries denote Good/Same/Bad counts.

Key findings: (1) RoboJudge effectively improves generation quality (Wan2.2 Bo5: 26 better vs. 4 worse, p < 0.001); (2) Bo5 consistently outperforms Bo3 in net gains; (3) RoboJudge achieves larger net gains than GPT-5.5 in 3/4 settings, consistent with our benchmark (overall r: 0.719 vs. 0.607).

These findings align with prior work on learned video judges as verifiers and rewards (Video-T1, VideoReward, VisionReward, LiFT, WorldModelBench). Using RoboJudge as a reward model for generator training is promising future work.

Weakness 3: Role of Rationale

Rationale supervision is important for both evaluation accuracy and OOD generalization, motivating our investment in human-annotated CoT. Rationale-based evaluation is also widely adopted (Prometheus 2, LLaVA-Critic, UnifiedReward-Think, VideoScore2). We ablate rationale supervision while keeping all other training settings identical.

SFTOverall rTotal MSEAvg. TokensLatency
Without rationale0.6900.8737.52.5s
With rationale (Ours)0.7000.78413414.0s
SFT (OOD, Official Prompt)VideoPhy2 PC rVideoPhy2 SA rPhysion JFormat Compliance
Without rationale0.0480.2750.0000/880
With rationale (Ours)0.2990.5520.105880/880

Rationale supervision significantly reduces MSE (−0.089, 95% CI excludes 0) and substantially improves OOD performance. Without it, physical correlation on VideoPhy2 drops to 0.048, and Physion format compliance collapses to 0/880. Thus, rationale supervision improves both judgment quality and instruction following, at the cost of approximately 5.6× inference latency.

AuthorsOpenReview ↗

Reply to Reviewer Z2vA

Dear Reviewer Z2vA,

As the discussion period draws to a close, we would like to follow up briefly on the three suggestions you raised. All are now addressed:

Real videos: top-end calibration improves (predicted PA=5: 16 → 65, human 91), correlation unchanged (0.701 → 0.700). Best-of-N: Wan2.2 Bo5 26 better / 4 worse (p < 0.001); larger net gain than GPT-5.5 in 3 of 4 settings. Rationales: without them, VideoPhy2 r drops 0.299 → 0.048, Physion format compliance 880/880 → 0/880.

If these address your concerns, we would be grateful if you would consider updating your assessment before the decision.

AuthorsOpenReview ↗

Gentle Follow-up on Our Rebuttal

Thank you again for your constructive feedback and helpful suggestions.

We would like to kindly follow up on our rebuttal responses. We have conducted additional experiments and analyses to address your concerns, and would greatly appreciate it if you could take a look when you have a chance.

Please let us know if you have any remaining questions or concerns. We would be happy to provide further clarification.

Thank you for your time and consideration!

Comments on the paper

1 comment
AuthorsOpenReview ↗

Global Comment

We thank all reviewers for their thoughtful feedback. Before addressing individual comments, we clarify our contributions and summarize the additional experiments conducted during rebuttal.

Scope and Contributions

Our work focuses on evaluating embodied video generation judges, with two main contributions:

  1. The first benchmark, to our knowledge, assessing judge alignment with human evaluations on Physical Adherence and Instruction Alignment.

  2. The first large-scale training dataset for embodied video judges, enabling the development of RoboJudge, a state-of-the-art 9B judge.

Our primary goal is alignment with human judgment. As an open-source 9B model, RoboJudge offers substantially lower costs and greater reproducibility than proprietary API-based judges. Evaluating judges through downstream policy performance is valuable, and we will discuss it as future work.

Additional Rebuttal Experiments

All experiments use a consistent inference protocol, with 95% bootstrap confidence intervals.

  1. Best-of-N Selection: We evaluate Wan2.2 (in-domain) and MiniMax-H3 (OOD) on 50 tasks each, with five independent human evaluators conducting blind GSB comparisons. RoboJudge-selected videos outperform one-shot generations (e.g., Wan2.2 Bo5: 26 better vs. 4 worse) and achieve larger net gains than GPT-5.5 in 3 of 4 settings.

  2. Rationale Ablation: Removing rationale supervision preserves in-domain correlation but increases prediction errors and severely hurts OOD generalization. Physical correlation on VideoPhy2 drops from 0.299 to 0.048, while Physion output-format compliance falls to 0/880.

  3. Real-Video Ablation: Adding 831 real demonstrations improves score calibration at the upper end: predictions of score 5 increase from 16 to 65 (91 human-labeled), without affecting overall correlation.

  4. Annotation Protocol: We provide additional support for our review-and-adjudication protocol from established benchmarks, including GPQA, TACRED Revisited, MMLU-Redux, and ADE20K.

We will incorporate these results and discussions into the revised paper.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.