- Rating
- 3(+2)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2(+1)
Read review
Paper summary
RubricEvo organizes rubrics into required criteria and alternative strategies, then evolves the graph when reward resolution or response diversity deteriorates. Validation and rollback mechanisms aim to maintain reward quality during evolution.
Critical strengths
Table 2 reports improvements across all 15 model–benchmark combinations. Appendix A.5 provides positive paired instance-bootstrap confidence intervals for every comparison. Table 3 directly tests graph structure while retaining online evolution. On Qwen3-1.7B Writing, the full method exceeds the evolving flat-rubric ablation by 1.58 points on WritingBench and 0.77 on Creative Writing. Blind human ranking preferences and WritingBench evaluation using a separate Qwen-7B critic provide supporting evidence beyond GPT-4o-based evaluation.
Critical weaknesses
The graph contribution is established in a limited setting. The controlled ablation covers one model size and domain. In the main comparison, EvoRubrics uses LoRA policy/generator adapters, whereas RubricEvo uses full-parameter policy training and an external GPT-4o evolver. These differences limit attribution of the broader gains specifically to graph structure.
GPT-4o both evolves rubrics and evaluates four of five benchmarks, creating possible shared-preference bias. Existing human and alternative-judge evidence mitigates, but does not eliminate, this concern.
BGE-M3 underlies both evolution diagnostics and diversity evaluation; the effective-mode analysis also uses these embeddings. An independent diversity measure would strengthen the claim that the method preserves distinct valid strategies.
Only one training seed is reported. Instance-bootstrap intervals do not quantify optimization variability. Numerical evolution-trigger thresholds, persistence settings, and rollback tolerances also require fuller specification.
Questions for authors
- Does the existing evolving-flat ablation reproduce its advantage in another domain or model size?
- Can the authors specify the evolution hyperparameters and quantify training-seed variability?
- Do the diversity conclusions hold under an independent representation or human assessment of distinct strategies?
7 comments
Q: What are the key evolution hyperparameters?
R: The appendix (Table 7) lists the training hyperparameters. Here we provide the key evolution settings. The reward-gap and diversity trigger thresholds are 0.85 and 0.95 of their initial values, with a quality floor of 0.55 and a persistence requirement of two consecutive audit windows. Rollback checks use a minimum retention of 50% of the predicted reward-gap gain and a maximum shadow-quality drop of 0.05. For diversity edits, mean diversity over two probation windows must improve over its pre-edit value by at least 0.5% in relative terms and 0.001 in absolute terms. We originally left these settings in the accompanying code and have now documented them explicitly in the revised appendix.
References
[1] EvoRubrics: Dynamic Rubrics as Rewards via Adversarial Co-Evolution for LLM Reinforcement Learning. §4.1, 2026.
[2] MA-RLHF: Reinforcement Learning from Human Feedback with Macro Actions. §4.3.1, ICLR 2025.
[3] DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models. §5.2.1, 2024.
We would be grateful if you would reconsider your assessment in light of the additional evidence, and we would be happy to address any further questions or concerns.
Thank you for the additional experiments. They broaden the evidence, but two central issues remain. First, please specify how Evolving Flat-RL is matched to RubricEvo in policy training, evolver, evolution opportunities, and validation budget; this determines whether Table R3 isolates graph structure. Second, the independent embedding evaluation demonstrates dispersion under another representation, but does not establish preservation of distinct valid strategies. The average advantage over EvoRubrics is modest (1.083 versus 1.061), lacks uncertainty, and reverses on WildBench. Please either provide direct support for the strategy-preservation interpretation or narrow that claim. The Claude results mitigate evaluator dependence, but their three ranking reversals and missing paired uncertainty also warrant more qualified performance claims.
I would appreciate additional evidence addressing these concerns before finalizing my scores.
Overall response. Thank you for your thoughtful feedback. To our knowledge, RubricEvo is the first work to jointly motivate rubric evolution by declining reward resolution and strategy diversity in open-ended RL. Its rubrics evolve alongside the policy as graphs that preserve shared requirements and give different valid strategies their own branches. We would be grateful for your reconsideration the score in light of the results below.
Q: How should we interpret the EvoRubrics comparison?
R: EvoRubrics’ dual-LoRA architecture is central to how its policy and rubric generator co-evolve [1]. We retained it to reproduce the baseline faithfully and compare the two methods fairly, with each retaining its defining technical choices.
Q: Reuse of GPT-4o
R: Claude Haiku 4.5 re-scored the same saved responses (Table R1).
Table R1. Claude Haiku 4.5 re-evaluation.
Although absolute scores shift systematically across judges, RubricEvo still achieves the best overall results under Claude Haiku 4.5.
Q: Does the diversity finding depend on BGE-M3?
R: Using 16 responses per prompt, we measure semantic dispersion with Qwen3-Embedding-8B. Table shows how the results vary across metrics and benchmarks. The results below show that RubricEvo still achieves the highest diversity score when evaluated with a newer, larger embedding model than the one used during training.
Q: Does the graph advantage extend across scales, domains, and seeds?
R: The original ablation used a fixed model size and domain, as in prior work [2, 3]. We extended the comparison to Qwen3-4B and Qwen3-8B on WritingBench and to Chat and Medical tasks at 1.7B. We use the official 7B WritingBench Critic for WritingBench and the same Claude Haiku 4.5 judge as above for all other benchmarks. Table R3 reports mean ± standard deviation across training seeds 42, 43, and 44. RubricEvo achieves higher mean scores than Evolving Flat-RL across the tested model sizes and domains.
Table R3. Controlled comparison across model sizes and domains.