- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces RewardVerse, a rubric-guided video reward modeling framework designed to mitigate "scalar drift"—the tendency of unconstrained multi-modal LLMs to produce collapsed or context-dependent scores. Instead of direct scoring, the method inserts a dynamic rubric generator that produces query-adaptive evaluation criteria (themes, weights, tips) without seeing the candidate video. A scorer then evaluates videos against these rubrics using a soft-logits readout to bypass natural language generation bias. The system is trained via Rubric-Guided Policy Optimization (RGPO), a two-stage GRPO-based algorithm that first warms up the scorer with offline seed rubrics and then jointly optimizes the generator and scorer using only 30 preference pairs per dimension. Experiments on the 16-dimensional EvalVerse benchmark and VGRB show state-of-the-art pointwise and pairwise performance, alongside downstream improvements in video generation.
Critical strengths
- The identification of scalar drift—specifically score-range collapse and high variance under context shift—as a core failure mode for video reward models is well-motivated. The initial 2x2 diagnostic study cleanly isolates the effects of rubric injection versus soft-logits decoding, providing solid empirical grounding for the subsequent architectural choices.
- Decoupling rubric generation from the candidate video is an elegant design choice. By conditioning the generator solely on the query, the framework structurally prevents the model from reverse-engineering lenient or strict criteria based on the video's visual quality, safeguarding evaluation objectivity while maintaining query adaptivity.
- The two-stage RGPO training pipeline is architecturally sound. Warming up the scorer with fixed seed rubrics before introducing dynamic generation avoids the highly under-determined joint optimization problem.
- Achieving competitive results with extreme data efficiency highlights the practical value of the rubric bottleneck as an inductive bias for low-data reward modeling.
Critical weaknesses
- The pairwise flip rate remains critically high (56.2%) even after rubric injection (Appendix A.1.6). While used to justify abandoning pairwise evaluation, such severe residual position bias calls into question the fundamental reliability of pointwise soft-logits extracted from the same underlying MLLM.
- Baseline comparisons rely on heuristic dimension mapping (Table 12). Since external baselines do not natively support all 16 EvalVerse dimensions, manually mapping their coarse outputs likely disadvantages them, making it unclear whether reported gains stem from the rubric mechanism or simply native alignment with the test taxonomy.
- Downstream RL validation is extremely narrow. Evaluating GRPO fine-tuning solely on Visual Quality (Section 5.4) fails to demonstrate whether dynamic rubrics generalize to optimizing complex temporal or narrative aspects like Logic or Pacing across the claimed 16 dimensions.
- Seed rubric synthesis relies entirely on Gemini-3.1-Pro. The paper lacks sensitivity analysis on how Stage 1 warm-up performs when the quality or diversity of these teacher-generated synthetic rubrics varies, leaving the student's performance ceiling poorly understood.
Questions for authors
- A detailed analysis of how the manual dimension mapping (Table 12) impacts baseline performance in Table 1, specifically whether evaluating baselines only on their native dimensions alters the overall ranking.
- Ablation results isolating the contribution of the soft-logits readout from the dynamic rubric during Stage 2 training, verifying whether continuous expectation extraction alone accounts for a significant portion of the PLCC/SRCC gains over natural language baselines.
- Downstream GRPO fine-tuning results on at least one cognitive or temporal dimension to validate that the query-adaptive rubrics effectively guide optimization beyond low-level visual quality.
- Sensitivity analysis regarding the seed rubric pool size and quality, clarifying how many Propose-Verify-Revise iterations or what minimum number of seed rubrics are required before Stage 1 scorer warm-up degrades.
- Clarification on the inference-time computational overhead introduced by the dynamic rubric generator, given that the scorer must now process structured JSON prompts containing multiple themes and tips for every video evaluated.
2 comments
Dear Reviewer,
Thank you again for your thoughtful and constructive feedback on our submission. We sincerely appreciate your time and effort throughout the review process.
As the November 18 discussion deadline approaches, we would like to kindly follow up on our previous responses and additional experiments. We hope they have adequately addressed your concerns.
If you have had a chance to review our responses, we would greatly appreciate knowing whether any concerns remain unresolved. We would be more than happy to provide further clarification or additional information before the discussion period ends.
Thank you once again for your time and consideration.
Best regards,
The Authors of Paper 3086
Thank you for calling the candidate-independent rubric design “an elegant design choice” and recognizing its data efficiency. We hope our response will resolve your concerns.
W1 — Pairwise position bias versus the deployed pointwise scorer.
The 56.2% pair-swap flip rate (Table 8) measures two-video A/B prompting, where swapping video positions can change the judgment. RewardVerse instead scores each video separately under the same query-only rubric. For this deployed pointwise scorer, Table 10 shows prompt sensitivity falling 0.206→0.085 after RGPO.
W2 & Q1 — Baseline mapping.
Because some Table 1 baselines reuse coarse scores across fine-grained dimensions, we also test on VGRB's VQ and TA tasks. RewardVerse ranks first among the cross-dataset baselines on both (Table 2). On EvalVerse, giving each baseline's native scoring heads equal weight changes PLCC/SRCC by at most 0.03/0.03, while RewardVerse remains first.
W3 & Q3 — Logic (D15) downstream RL.
We added Logic-guided downstream GRPO (full results in our response to ShwE, W1 & Q1). Compared with the VideoReward-rewarded arm, RewardVerse achieves higher StoryEval event completion (0.9524 vs. 0.8988), DreamRunner transition scores (0.7467 vs. 0.7223), and the mean of four EvalVerse-based Logic sub-aspects (4.183 vs. 3.817), extending downstream validation to temporal and narrative quality.
W4 & Q4 — Seed-rubric sensitivity.
Seed quality: With themes and weights fixed, reducing tips from four to one per theme lowers Stage-1 PLCC/SRCC from 0.456/0.364 to 0.442/0.328. PVR convergence: Propose–verify–revise converges within four rounds, with 97.9% of accepted candidates requiring at most one revision. Seed quantity: From a candidate pool capped at 200 per dimension, MCR² shortlists five representative rubrics; we only select one highest-quality rubric per dimension for Stage-1 training. We have clarified this in paper.
Q2 — Soft-logits vs. dynamic rubrics.
Continuous expectation extraction alone does not explain the Stage-2 gains. The rubric-free scorer trained with soft-logits reaches 0.439/0.351 (Table 3e). With soft-logits unchanged, continuing scorer training under fixed rubrics reaches 0.467/0.380, while learning dynamic rubrics under the same Stage-2 budget reaches 0.530/0.416:
With soft-logits held fixed, the +0.063 PLCC / +0.036 SRCC gain over the matched control supports the benefit of learning dynamic rubrics.
Q5 — Measured rubric inference overhead.
Table 15 reports 1.7 s/video for rubric-free soft-logits and 2.0 s/video for rubric-guided soft-logits (+0.3 s, approximately 18%). The generator produces all five themes and tips in one JSON rubric per query; the scorer then obtains all five theme scores in a single completion per video, not five separate forward passes. Thus, for one candidate, the pipeline needs one rubric-generation call and one scoring call. The rubric is reused across candidates sharing a query, amortizing the generation cost as Tgen/N.