- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper examines methods for turning non-verifiable problems (such as video script rating) into verifiable problems, by using an external model (GPT5.4) to generate a rubric to score the outputs. In this case the rubric evolves as particular criteria saturate, allowing the model to continually improve on distinct dimensions. This is quite simple and neat; the authors run extensive experiments with appropriate baselines, comparing criterion variants that exhibit desirable properties or avoid flaws (the latter "negative" works best). They also Claude / GPT as raters and find that negative beats sample static (a control in without criterion evolution) quite handily; human raters provide a similar result.
I thought this was quite a neat paper. It's a simple, clear innovation that is nicely explored, well tested, and clearly reported.
Critical strengths
Very simple idea; nicely implemented. Elegant experiments, clearly reported.
Critical weaknesses
In this Rubric evaluation literature I struggle with the question of circularity; for example here GPT5.4 is the rubric evaluators, as well as (one of) the eventual judge(s). Nice to have human validation, but what the actual ground truth here?
Questions for authors
Does this work for domains other than the one tested?
Follow-up: Additional Evidence for RARE’s Criterion Selection and Adaptation
Dear Reviewer AXy1,
We would like to follow up on our earlier message and highlight two findings that strengthen the evidence for RARE’s design choices:
Full-score saturation outperforms alternative selection rules under matched update and compute budgets. In the controlled comparison on the 3D / Stop-motion domain, averaged over two training seeds, RARE outperforms zero-variance and random selection by 6.24 and 7.46 points, respectively. These comparisons isolate the contribution of the selection rule: full-score saturation replaces criteria that all current responses already satisfy, while preserving criteria that capture requirements the policy has yet to master.
New criteria provide useful feedback beyond the triggering batch, while the policy largely retains previously satisfied requirements. Our concrete example illustrates how a new criterion captures observable, request-relevant quality differences distinct from those assessed by retained criteria. Across 940 first-round update events, the new criteria still distinguish later-checkpoint responses in 43.0% of events at the end of training. Meanwhile, 90.4% of newly sampled rollouts still receive full marks on the replaced criteria, supporting the value of introducing new feedback while preserving previously learned behavior.
Together, these results support RARE’s central contribution: selectively replacing fully satisfied criteria with new, informative criteria allows training feedback to adapt as the policy improves.
We would appreciate your assessment of this additional evidence. If it addresses your concerns, we would be grateful if you could consider revising your score upward to reflect the strengthened empirical support for our contribution. Please let us know if you have any remaining concerns so that we can address them during the discussion period.
Thank you again for your time and consideration.
Best regards,
The Authors