- Rating
- 3
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
-
The paper argues that rubric-based RL often optimizes recall through rubric coverage, without adequately penalizing unnecessary or incorrect content, creating reward hacking behaviors
-
The authors divide responses into sentence-like units and estimate precision using rubric-based necessity credits and LLM judgments of factual correctness. They introduce the FORK to balance precision and recall accordingly
-
Experiments compare FORK against standard rubric-RL, OnlineRubrics, universal rubrics, and holistic preference rewards. Results show higher F1 and GPT-5.4-judged win rates against the baseline methods
Critical strengths
-
Creating absence-based rubrics to balance precision and recall is difficult, and the paper innovatively introduces unit-level judgments to address this problem through structured necessity and factuality checks on responses from a stronger judge. This is an intuitive and practical approach that help address the recall-focused optimizations in rubric-RL
-
Solid experiment and ablations results across healthcare and science domains, supported by precision, recall, and win rates judged by a stronger LLM.
Critical weaknesses
-
While the authors demonstrate improvements in F1 under a separate evaluation grader, FORK's optimizes an objective built from closely related components, making it less clear how much these gains reflect quality improvements rather than alignment between the reward shaping and eval metrics. More qualitative analysis of the where the improvement comes against baseline methods (OnlineRubrics, Holistic Preference) from would strengthen author's argument
-
Related to the first point, win rate provides a quality metric less directly tied to reward shaping, but the rubric-free GPT-5.4 judge may miss task-specific requirements in these specialized domains. A rubric-informed preference evaluation or any audit of the judge’s ability to evaluate those tasks would help clarify
Questions for authors
As commented in the weakness section, I'm wondering if there's any more qualitative analysis against baseline methods (OnlineRubrics, Holistic Preference) on where the improvements come from, and whether the win rate holds for a rubric-informed judge.
We thank the reviewer for the constructive and detailed comments.
Q1. I'm wondering if there's any more qualitative analysis against baselines (OnlineRubrics, Holistic Preference) on where the improvements come from.
A. OnlineRubrics slightly improves the rubric coverage, but since it is still a recall-based objective, the response contains unnecessary content and are verbose. We also observe the same phenomenon as the standard coverage reward where the model first generates an initial response, add another response, and repeat this process. Holistic Preference is not verbose, but the response shows low rubric coverage and are less specific. Compared to these two baselines, FORK mostly produces necessary contents and are less verbose, while retaining the rubric coverage. We add model responses of OnlineRubrics and Holistic Preference in pages 18-20 in Appendix D.1. The response of FORK is already provided in page 15.
Q2. Does the win rate hold for a rubric-informed judge?
A. We adapt the rubric-free judge prompt to include rubric coverage as one of the evaluation criteria. Specifically, the judge prompt originally considers five dimensions (completeness, factual correctness, conciseness, relevance, and safety) and we replace the completeness dimension into the rubric coverage. The following table shows that FORK still outperforms baselines in terms of rubric-informed win rate against the base policy.