acceptodds
Under review as a conference paper at ICLR 2027

Video-Rubrics: Fine-Grained Rewards for Faithful Embodied Video Generation

Abstract

Video diffusion models have made substantial progress in visual quality, yet instruction adherence remains challenging in embodied video generation. Semantic rewriters enrich the conditions supplied to video generators, but rewards based only on rewritten requirements can favor plans that omit difficult parts of the original request. We introduce Video-Rubrics, a reinforcement learning framework that jointly optimizes a prompt rewriter and a video generator through fine-grained visual criteria. We construct Video-Rubrics-60K, a training set of image-text requests spanning general scenes and robotic instructions, with atomic requirements annotated independently of candidate rewrites. Original-prompt rubrics supervise both requirement preservation and visual realization for the rewriter, while condition rubrics evaluate how faithfully the generator follows its actual input. A frozen vision-language verifier assesses each requirement using timestamped video evidence, with soft temporal windows that preserve action context. We normalize item-level rewards within each policy's comparison groups and use the resulting advantages to update both policies from the same sampled videos. Experimental results show that Video-Rubrics improves instruction adherence across three video backbones, with gains on multi-entity interactions and long-horizon tasks.

Primary area:
generative models
Keywords:
Video Generation, Reinforcement Learning, Reward Models

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_aKYTOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

A scalar reward on a whole video cannot say which requirement the video failed, so the paper builds a fine-grained rubric reward for RL post-training. Its core design is a two-rubric architecture: an original-prompt rubric decomposed offline by Gemini 3.7-Flash into binary items over six requirement categories and frozen before training independently of the sampled rewrites, plus a per-rewrite condition rubric compiled by a frozen Qwen3.8-27B from the condition actually fed to the video model with a coverage check; items are scored by a frozen VLM's yes/no logit difference over a soft temporal window, zeroed unless prerequisites pass, and turned into item-wise group-relative advantages driving one on-policy GRPO step on the rewriter and one coefficients-preserving-sampling step on the video policy.

It also contributes Video-Rubrics-60K and trains three backbones (LingBot-Video 30B-A3B, Cosmos3-Nano, MiniMax-H3) for 1000 iterations. Evaluation is on RBench only: Overall 0.614 → 0.625, 0.584 → 0.591, 0.660 → 0.669, plus ablations over the update strategy and the window policy.

Critical strengths
  • Freezing the original-prompt rubric independently of the sampled rewrites is the paper's best idea, and it answers a failure mode that quietly ruins most rubric-based RL.

  • The serializer/compiler split and its fixed checkpoints; the item schema with source span, grounding flag, evidence scope and prerequisites.

Critical weaknesses
  • The measured effect is smaller than the unreported noise, and the per-category pattern contradicts the paper's own motivation. RBench Overall moves by +0.011, +0.007 and +0.009. The paper reports no seeds, no standard deviations, no error bars, no confidence intervals and no significance tests anywhere.

  • The central architectural claim is never ablated, the verifier is never validated, and the Base rows appear to be copied rather than re-run. Three gaps compound: 1) No anchoring ablation. There is no experiment comparing the frozen original-prompt rubric against one compiled only from the rewrite. The two ablations that exist — update strategy (Table 2) and window policy (Table 3) — are both orthogonal to the design decision identified above as the main contribution. So strength (1) is asserted, not demonstrated. 2) No verifier validation. The reward is a yes/no logit difference from a frozen Qwen3.8-27B. There is no agreement with human labels, no precision/recall on the binary items, no per-category scorer reliability, and no reward-hacking analysis — despite 1000 iterations of RL against exactly that scorer, and despite the rewriter being explicitly rewarded for coverage of items it does not itself produce. The hard 0.5 gating threshold adds an unmeasured discontinuity. 3) Base rows. App. E.1 states the nine Base category scores "are retained with their reported precision" from published entries. That sits uneasily with Sec. 3.1's claim that Base and Video-Rubrics "use the same evaluation inputs and generation settings", and with a +0.011 headline delta: if Base comes from a different setting or rounding, the delta is not like-for-like.

Questions for authors

How accurate is the frozen item scorer? A human-labeled subset (even 300–500 items) with per-category precision/recall, or item-level agreement with the RBench evaluator, would let me judge whether the reward measures what it claims. Also: how sensitive are results to the hard 0.5 gating threshold?

Reviewer_JpAmOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies joint post-training of a prompt rewriter and a video generator to improve instruction adherence in embodied video generation. Its central design uses fixed original-prompt rubrics to evaluate the rewriter and condition-specific rubrics to evaluate the generator. The rewriter reward combines visual realization of original requirements with a penalty for omitted requirements, while the generator reward measures execution of the rewritten condition. A frozen vision-language model supplies item-level scores, and the two policies are updated using GRPO and a CPS selected-transition policy-gradient objective, respectively. The authors construct Video-Rubrics-60K and report modest improvements in RBench Overall scores across three video backbones.

Critical strengths

The paper identifies a plausible failure mode: evaluating only rewritten requirements may incentivize the rewriter to simplify the task by omitting difficult steps. Anchoring evaluation to fixed original requirements is a targeted response. Separately checking actions, completion states, and temporal relations is appropriate for embodied videos, where depicting a relevant action does not necessarily imply successful execution.

Critical weaknesses

The central reward design is not directly validated. Tables 2–3 examine update strategies and temporal windows, but do not isolate the contribution of dual rubrics over a single-rubric alternative or test removal of the omission penalty in Eq. 2. The paper also does not quantify the motivating phenomenon of requirement omission. Consequently, the results do not establish whether the gains arise from the proposed omission-prevention mechanism. The modest gains lack uncertainty estimates. Overall improvements in Table 1 are 0.011, 0.007, and 0.009. Joint training exceeds alternating training by only 0.002 in Table 2, and the comparison does not match optimizer-step counts. Without confidence intervals or repeated-run variability, the robustness of these differences is unclear. The comparability of the main-table baselines is unclear. Section 3.1 states that Base and the proposed method use the same initial rewriting pipeline and generation settings. However, Appendix E.1 retains some published baseline scores, while Appendix C.1 notes that published baselines retain their native settings. The authors should clarify which Table 1 baselines were rerun under a fully matched pipeline before attributing the gains to the proposed training method.

Questions for authors

Table 2 compares update strategies under the same video-generation budget, but joint training performs more optimizer updates. Can the authors clarify whether its benefit comes from co-adaptation between the rewriter and generator or from additional parameter updates? Does the advantage persist against alternating training with a matched update budget, or sequential training that optimizes one policy before the other? This distinction is important for establishing the benefit of joint over separate optimization. The method uses the verifier’s binary-answer probabilities as item-level soft rewards and a threshold of 0.5 to gate dependent items. To what extent do these scores agree with human judgments and reflect verifier uncertainty, particularly for action completion and temporal relations inferred from sparsely sampled frames? Have the authors evaluated reliability by requirement type and sensitivity to the gating threshold to determine whether upstream verification errors systematically suppress credit for correct downstream behavior?

Reviewer_S77MOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper presents Video-Rubrics, a reinforcement learning framework for improving instruction adherence in embodied video generation. The method jointly optimizes a prompt rewriter and a video generator through fine-grained visual rewards. It introduces fixed original-prompt rubrics to preserve user requirements and condition rubrics to evaluate the generated videos against rewritten instructions. A frozen vision-language model evaluates individual requirements using temporally localized video evidence, and item-wise advantages are used to update the two policies through GRPO and CPS, respectively. The authors also construct Video-Rubrics-60K, containing 60K image-text requests with atomic annotations. Experiments on RBench across three video generation backbones demonstrate improvements in instruction adherence.

Critical strengths
  1. The paper presents a reasonable framework for jointly optimizing prompt rewriting and video generation with fine-grained rewards.

  2. The experiments cover three video generation backbones, and the results show improvements across different model architectures.

Critical weaknesses
  1. Limited novelty: The framework mainly combines existing rubric-based rewards, VLM verification, GRPO, and CPS. Although the dual-rubric design is reasonable, the underlying optimization techniques are largely inherited from prior work.

  2. Limited performance gains: The overall improvements on RBench are relatively modest (0.007–0.011 across three backbones).

  3. Insufficient validation of key components: The ablations mainly focus on policy update strategies and temporal windows. The individual contributions of the dual-rubric design, coverage penalty, and item-wise normalization remain unclear.

Questions for authors
  1. Could the authors provide additional ablations to isolate the contributions of the dual-rubric design and item-wise reward normalization?

  2. How well do the automatically generated rubrics and VLM verification scores agree with human judgments, particularly for complex multi-step actions?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.