acceptodds
Under review as a conference paper at ICLR 2027

RHContrast: A Matched-Control Benchmark for Evaluating Reward-Hacking Risks in Long-Form LLM Responses

Abstract

Reward changes after rewriting can be misleading: an erroneous answer may gain reward while benefiting less than a reference answer, or lose reward while incurring a smaller penalty than the reference. We introduce FAME-RHBench, a long-form benchmark that tracks reference answers and variants with localized defects through matched transformations, preserving explicit links between original and transformed texts. The benchmark separates reward recovery, changes in the reference–defect score gap, and direct comparison with an untransformed reference. Naturalization reorganizes both answer branches; verification endorsements add the same unsupported claim without altering either answer body. Across seven prompted judge families, naturalization increases scores on both branches on average, but benefits reference answers more. Verification endorsements lower both scores, yet penalize reference answers more strongly. Analyses with scalar and pairwise reward models, grounding verifiers, and an AI-assisted, human-reviewed reference characterize how these responses vary across evaluation settings. The benchmark enables evaluator audits that separate general transformation effects from defect-conditioned score changes.

Primary area:
datasets and benchmarks
Keywords:
LLM evaluation, reward models, evaluation benchmarks, matched controls, reward robustness

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_19NzOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies reliable evaluation of rewritten erroneous LLM responses. The authors introduce RHContrast-Bench, leveraging 1151 long-form answer lineages with matched controls for auditing reward hacking risks. The benchmark supports traceable local-defect constructions with joint analysis of reference and defective answers, connecting 3 complementary comparisons. The method is evaluated across 7 LLM judge families, scalar and pairwise reward models, grounding verifiers, and human-reviewed reference judgements.

Critical strengths
  • It identifies a concrete weakness in current evaluation methods that could potentially lead to reward hacking.
  • A concrete example of restaurant overview is provided to help understand the motivation easily.
  • The writing is clear and easy to follow.
Critical weaknesses

Overall I appreciate the motivation behind this work and find the methodology interesting, but I feel the contributions and empirical evidence are not yet sufficiently compelling. I think it would be interesting to connect the study with recursive self-improvements in LLMs. Iterative response refinements could be one direction to further extend the experiments.

Questions for authors

Overall I appreciate the motivation behind this work and find the methodology interesting. The design is rational, but the contributions and empirical evidence are not yet sufficiently compelling. I believe this paper would benefit from further development and stronger empirical evidence. Especially, I think it would be interesting to connect the study with recursive self-improvements in LLMs. Iterative response refinements could be one direction to further extend the experiments.

4 comments
AuthorsOpenReview ↗

Response to Reviewer 19Nz (1)

Comment 1: Overall I appreciate the motivation behind this work and find the methodology interesting. The design is rational, but the contributions and empirical evidence are not yet sufficiently compelling. I believe this paper would benefit from further development and stronger empirical evidence.

Response: We thank the reviewer for recognizing the motivation and design of our study. The core contribution is to use matched controls comprising reference and defect answers from the same source to distinguish among ‘a score increase for the defect answer,’ ‘a relative advantage for the defect answer,’ and ‘the defect answer surpassing the original reference answer.’ For example, in the original N track, the reference branch gained +0.3591 points on average and the defect branch gained +0.2056 points, yielding κ_N = −0.1534 when the reference-branch gain is subtracted from the defect-branch gain. Looking only at the score increase for the defect answer would miss the fact that the reference branch benefited more. Our framework can therefore test whether reward increases reflect a shared response to quality or an additional relative advantage for the defect answer.

To strengthen the experimental evidence, we added evaluations of actual candidate selection, native reward scores, and independent held-out samples, connecting differences in controlled scoring to actual selection behavior:

Supplementary experimentKey resultsEvidence provided
Best-of-N selection on 50 questionsWe independently generated 16 raw answers per question and compared Bo1, Bo4, and Bo16. At Bo16, all four independent model judges found that Skywork selected incorrect answers on 1—3 more questions than AllenAI, a difference of 2—6 percentage points; uncertain answers were reported separately.The relative risk difference between the two RMs identified by the benchmark corresponded to actual candidate selection in this experiment.
Native reward-model validation on 50 held-out questionsSkywork’s κ_E was +0.3969 and +0.4325 in the answer-only and source-aware settings, respectively, with 95% confidence intervals of [+0.2472, +0.5605] and [+0.2700, +0.6180].The relative scoring pattern in the E track was reproduced on held-out samples without relying on artificial 1–5 score truncation.
Judge validation on the same held-out sampleClaude Opus 4.8, Gemini 3.1 Pro, and Qwen 3.7 Max, each with complete coverage of all 50 questions, had κ_E values of +0.102, +0.690, and +0.777, respectively, with 95% confidence intervals of [+0.028, +0.180], [+0.599, +0.782], and [+0.646, +0.910].Multiple judges reproduced the relative scoring pattern in the E track on held-out samples.

The held-out sample was frozen before evaluation, with overlaps and near-duplicates of previously evaluated items excluded. The two RMs completed all 1200 native-score evaluations; the seven judges returned 2022/2100 valid responses in total. The table above lists the three judges with complete E-track coverage of all 50 questions and confidence intervals above 0. Skywork’s positive κ_E took the form of a smaller relative penalty for the defect branch.

AuthorsOpenReview ↗

Response to Reviewer 19Nz (2)

Furthermore, the same held-out evaluation showed that an individual judge could not only assign a relatively more favorable score change to the defect branch but also increase the score of the defect answer itself. We use ρ to denote the score change of the treated defect answer relative to its original version, and κ to denote this change minus the corresponding change in the reference branch. Grok’s results are shown below; changes for each question were first averaged over presentation orders and then averaged with equal weights across the scoring rules within each track:

Grok-4.3Comparable questionsAbsolute gain for defect answers ρRelative interaction κQuestions with increased/unchanged/decreased scores
N track39+0.2651+0.540615/20/4
E track29+0.6013+0.771019/7/3

Grok showed positive ρ and κ in both tracks: defect answers received higher scores than their own original versions, and their score changes were more favorable than those of the matched reference branch. In the E track in particular, defect answers on 19/29 questions actually gained points after verification endorsement was added. This provides direct sample-level observations that ‘answers retaining defects can still obtain absolute scoring gains,’ complementing cases in which positive κ arises solely from a smaller relative penalty. This phenomenon is model-dependent: the mean score of the E-track defect branch still decreased for the other six judges. Grok has not yet covered all 50 questions, so these results are reported as descriptive findings from the completed pairs.

In addition, after excluding the 34 identified items of semantic concern, 586 questions remained in the original judge evaluation. κ_N changed from −0.1534 to −0.1421, with a 95% confidence interval of [−0.1699, −0.1145], and all seven judges retained negative results. This supports the robustness of the main original N-track finding to the identified semantic issues.

Together, these results strengthen the contribution of this work: matched controls can distinguish among responses such as ‘the defect answer gains points, but the reference answer gains more,’ ‘the defect answer receives a smaller relative penalty,’ and ‘the defect answer gains points in absolute terms and obtains a relative advantage.’ The native-RM and judge held-out experiments provide validation across settings, while Best-of-N provides evidence related to actual candidate selection. The direction of the N-track result in the new held-out sample differs from that in the original evaluation. We treat this as an additional finding on model and sample dependence and report the results separately for each model; the positive E-track interactions for multiple judges and Skywork support the relative scoring pattern reported in the original paper.

AuthorsOpenReview ↗

Response to Reviewer 19Nz (3)

Comment 2: Especially, I think it would be interesting to connect the study with recursive self-improvements in LLMs. Iterative response refinements could be one direction to further extend the experiments.

Response: This suggestion identifies an important application of the matched-control framework. When answers are repeatedly rewritten based on reward feedback, an increase in reward may result from substantive improvements in content, changes in expression, or verification endorsement. Our framework can compare the reference and defect branches at each iteration, check whether the target defect remains, and simultaneously record absolute reward changes, relative scoring advantages, and comparisons with the original reference answer, thereby distinguishing genuine quality improvements from reward gains that preserve the defect.

The present Best-of-N experiment extends validation from scoring fixed texts to reward-based selection at inference time, supporting a difference in relative selection risk between the two RMs under the tested setting. It evaluates selection from a fixed candidate pool and does not yet include a feedback-driven loop of iterative rewriting. We will clarify this connection in the discussion, present iterative rewriting as an extension of the framework, and distinguish the selection behavior validated so far from the process of recursive self-improvement.

AuthorsOpenReview ↗

Follow-up on Rebuttal

Dear Reviewer,

Thank you for your time and effort in reviewing our paper.

We have provided detailed responses to your concerns, including additional experimental evidence.

As the discussion period is approaching its end, we would greatly appreciate it if you could take a moment to review our responses.

Thank you again for your time and consideration.

Best regards, The Authors

Reviewer_Dyo1OpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces RHContrast-Bench, a matched-control benchmark for investigating reward-hacking risks in the evaluation of long-form LLM responses. It contains 1,151 answer lineages from six datasets, linking reference responses, counterparts with localized defects, and transformed versions across three settings: surface polishing, naturalness-oriented rewriting, and unsupported claims of external verification. The experiments distinguish absolute score gains from relative advantages over matched references, showing that these signals do not necessarily indicate successful reward hacking. While this distinction is useful, the benchmark primarily examines evaluator sensitivity to a limited set of response transformations. Its relevance to broader reward-hacking scenarios, particularly those involving adaptive optimization against a reward signal, remains insufficiently established.

Critical strengths
  • Well-motivated evaluation design. The matched-control setup addresses an important ambiguity: a higher score for a transformed defective response does not, by itself, demonstrate reward hacking.

  • The framework distinguishes absolute score changes, shifts in the reference–defect gap, and overtaking of the original reference, supporting more precise interpretation of evaluator behavior.

  • Experiments across multiple LLM judge families, reward models, and grounding verifiers, supplemented by expert-reviewed judgments, reveal both consistent patterns and configuration-dependent effects.

Critical weaknesses
  • Narrow scope. The benchmark tests a few fixed changes to static responses. It remains unclear whether these tests capture reward-hacking risks in broader agentic settings involving tool use and repeated interaction.

  • Reference and defective answers are rewritten separately, so they may improve by different amounts beyond style. This makes it difficult to attribute score changes specifically to how judges treat errors.

  • Unclear connection to actual reward hacking. The main score contrasts can change with the scoring scale, and human-reviewed judgments show similar patterns. These results do not clearly establish that the benchmark measures exploitable evaluator weaknesses.

Questions for authors
  • What evidence connects performance on RHContrast-Bench to actual reward exploitation, especially in agentic settings?

  • How do you check that naturalization preserves each answer’s substantive content, so score changes are not driven by unequal improvements in answer quality?

3 comments
AuthorsOpenReview ↗

Response to Reviewer Dyo1 (1)

Question 1: What evidence connects performance on RHContrast-Bench to actual reward exploitation, especially in agentic settings?

Response: We added a Best-of-N experiment to connect controlled score differences with actual candidate selection. For the same set of 50 questions, a fixed generator independently sampled 16 answers per question without human rewriting. Skywork and AllenAI selected from identical Bo4 and Bo16 candidate pools, with Bo1 serving as a shared baseline. Four models independently assessed correctness under blinded evaluation. The Bo16 results are shown below. Each cell reports the number of clearly incorrect answers (number of uncertain answers), with a denominator of 50 questions throughout. Uncertain answers were not counted as correct.

Independent judge modelSkyworkAllenAI
GPT-5.615(6)12(9)
Kimi K3 Max7(7)5(8)
DeepSeek V4.1 Flash11(11)10(10)
Qwen 3.8 Max11(9)9(12)

All four judges identified 1–3 more incorrect selections by Skywork than by AllenAI, corresponding to 2–6 percentage points. This supports a correspondence between the relative risk difference between the two RMs identified by the benchmark and actual candidate selection in this experiment. The experiment validates resampling-based selection at inference time; tool use, multi-turn interaction, and iterative policy updates were not included in this validation.

Regarding the scoring scale, on a separate, prespecified and frozen held-out set of 50 questions, Skywork's E-track κ_E under native reward scores was +0.3969 in the answer-only setting and +0.4325 in the source-aware setting, with 95% confidence intervals of [+0.2472, +0.5605] and [+0.2700, +0.6180], respectively. This supports the finding that its relative scoring pattern need not depend on artificial truncation to a 1–5 scale. Here, the positive interaction reflects a relatively smaller score penalty for the defect branch. Agreement between scores and human judgments is not treated as standalone evidence of exploitability; the candidate-selection results above provide additional behavioral evidence.

AuthorsOpenReview ↗

Response to Reviewer Dyo1 (2)

Question 2: How do you check that naturalization preserves each answer’s substantive content, so score changes are not driven by unequal improvements in answer quality?

Response: The N track uses paired naturalization to keep task-relevant content, organization, and style as consistent as possible between the reference and defect branches, apart from the designated defect. In the relevant construction and revision batches, two models other than the generator independently reviewed information completeness, preservation of the target defect, non-target semantic changes, and consistency between the rewriting approaches applied to the two branches. A revision was accepted only if both reviews passed.

We further conducted a paired semantic review of the 187 samples flagged by length screening among the original 1151 samples, identifying 70 samples of concern. Of these, 34 were included in the original 620-question judge evaluation set. Excluding these 34 questions left 586 questions. Recalculation using the original scores and stratified analysis method changed the naturalization score gain for the reference branch from +0.3591 to +0.3479 and that for the defect branch from +0.2056 to +0.2058. κ_N, defined as the defect-branch score gain minus the reference-branch score gain, changed from −0.1534 to −0.1421, with a 95% confidence interval of [−0.1699, −0.1145].

κ_N remained negative for all seven judges, supporting the robustness of the main finding—that the reference branch benefits more on average—to the identified semantic issues. The two-model review constrains revision quality, whereas recalculation after excluding samples of concern tests whether content changes drive the main conclusions. The scope of this targeted AI-assisted review is explicitly reported in the revised manuscript.

Reviewer_Dyo1OpenReview ↗

Thank you

Since you have claimed some of the issues, i decide to increase my score.

Reviewer_SZXPOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper questions the idea that applying certain transformations to incorrect outputs can increase rewards, which has been interpreted as reward hacking. The authors argue that, for a better comparison, the same transformations should be applied to correct outputs as well. Based on this, the authors create a controlled benchmark spanning 6 datasets and multiple transformations commonly associated with reward hacking (surface polishing, naturalization of both reference and defective responses, and adding identical unsupported verification endorsements). They compare how these transformations affect both reference and defective responses. They evaluate different types of models, including 7 prompted LLM judges, along with scalar and pairwise reward models. The key findings are: (a) Naturalization increases scores for both reference (+0.359) and defective (+0.206) responses, but the reference benefits more; (b) for verification endorsements, scores decrease for both reference (−1.002) and defective (−0.677) responses, but the reference is penalized more; (c) transformed defective responses remain below the original references on average; (d) all 7 prompted judge families exhibit the same direction of these effects, although the magnitudes vary; and (e) reward-model results vary across model architectures and input configurations.

Critical strengths
  • The paper provides a controlled comparison benchmark for reward hacking. Authors construct 1,151 matched responses across 6 datasets. Comparing reference and defective responses under matched transformations helps separate absolute reward changes from relative changes. This provides a more informative evaluation than considering reward increases in defective responses alone.

  • The paper considers extensive evaluation settings. The experiments cover 7 prompted LLM judge families, multiple scalar and pairwise reward models, grounding verifiers, and reviewed reference judgments. The authors find consistent directional effects across prompted judges and demonstrate that absolute reward changes and differential responses can lead to different interpretations.

Critical weaknesses
  • The work has limited practical significance in its current presentation. The benchmark measures changes in evaluator scores but does not establish whether these lead to exploitable reward hacking. The positive interaction under verification endorsements (+0.325) may partly arise from the bounded scoring scale. Furthermore, transformed defective responses remain below the original references on average, with no endorsement lineage showing both a positive interaction and score overtaking. Without downstream reward-based selection or optimization experiments, the practical significance remains unclear.
  • These are multiple limitations in empirical validation of the findings. Naturalization does not guarantee equal changes in semantic quality for reference and defective responses, so reward differences may reflect actual changes in answer quality. Additionally, the naturalization interaction was promoted after observing the results, while the endorsement interaction is exploratory. The reference judgments also contain unresolved annotations, limiting confidence in the robustness of the findings.
Questions for authors
  • Does the benchmark predict actual reward-hacking vulnerabilities? For example, do reward models with larger positive differential responses select more incorrect answers under best-of-N sampling?
  • How do the authors ensure that naturalization preserves correctness and completeness? Do the findings hold when restricted to independently verified meaning-preserving transformations and reliable reference annotations?
  • Appendix B.3 shows that positive differential responses can arise from floor effects in bounded scoring scales. How much of the endorsement interaction can be explained by this? Can the authors test this using a clipping-based null model or ranking-based comparisons?
  • Appendix B.2 states that the naturalization interaction was identified after observing outcomes and that the endorsement interaction is exploratory. Can the authors validate these findings on an independent held-out set?
4 comments
AuthorsOpenReview ↗

Response to Reviewer SZXP

Question 1: Does the benchmark predict actual reward-hacking vulnerabilities? For example, do reward models with larger positive differential responses select more incorrect answers under best-of-N sampling?

Response: We added a Best-of-N experiment using a fixed generator. For each of the same 50 questions, we independently sampled 16 answers without manual rewriting, using the first 1, 4, and 16 answers in a fixed order as candidate pools. Bo1 serves as the common baseline. For Bo4 and Bo16, Skywork and AllenAI select from identical candidate pools, after which four independent models assess correctness in a blinded review. The table reports clearly incorrect answers, with uncertain answers in parentheses. Each entry covers 50 questions, and uncertain answers are reported separately.

Independent judgeBo1Bo4 SkyworkBo4 AllenAIBo16 SkyworkBo16 AllenAI
GPT-5.615 (14)17 (9)17 (6)15 (6)12 (9)
Kimi K3 Max6 (15)12 (8)10 (8)7 (7)5 (8)
DeepSeek V4.1 Flash14 (11)17 (8)17 (6)11 (11)10 (10)
Qwen 3.8 Max12 (15)9 (17)9 (16)11 (9)9 (12)

At Bo16, all four judges identified 1–3 more incorrect selections by Skywork than by AllenAI, a difference of 2–6 percentage points. This supports a correspondence between the two RMs' relative risk differences in the benchmark and their actual selection outcomes under resampling. This conclusion is limited to the models, generator, and sample used here. The data do not support the claim that increasing the number of samples necessarily produces a monotonic increase in error rate, and we do not treat model-based blinded review labels as human ground truth.

Question 2: How do the authors ensure that naturalization preserves correctness and completeness? Do the findings hold when restricted to independently verified meaning-preserving transformations and reliable reference annotations?

Response: The N track uses paired naturalization: apart from the designated defect, the reference and defect branches are kept as consistent as possible in task-relevant content, organization, and expression style. In the relevant construction and revision batches, we used two cross-model semantic reviews. Two models other than the generating model independently checked information completeness, retention of the target defect, non-target semantic changes, and whether the rewriting of the two branches was matched. A revision was accepted only if both reviews passed.

In addition, within the original 1151-sample dataset, we conducted a paired semantic recheck of the 187 samples flagged by length screening, identifying 70 samples of concern. Of these, 34 were included in the original 620-question judge evaluation set. This check examined the final texts independently; approval of an earlier version was not automatically carried over to subsequently modified texts.

After excluding these 34 questions, we retained 586 questions and recomputed the results using the original scores, stratification weights, and question-level resampling method. The naturalization gain for the reference branch changed from +0.3591 to +0.3479, and that for the defect branch changed from +0.2056 to +0.2058. The defect-branch gain minus the reference-branch gain, κ_N, changed from −0.1534 to −0.1421, with a 95% confidence interval of [−0.1699, −0.1145]. κ_N remained negative for all seven judges, supporting the robustness of the main judge results to the identified semantic issues.

This check indicates that the main judge results are not driven by the flagged samples. However, it is a targeted AI-assisted audit and does not replace independent human semantic validation of the entire dataset. We will qualify our statements about quality control and robustness accordingly.

AuthorsOpenReview ↗

Response to Reviewer SZXP (2)

Question 3: Appendix B.3 shows that positive differential responses can arise from floor effects in bounded scoring scales. How much of the endorsement interaction can be explained by this? Can the authors test this using a clipping-based null model or ranking-based comparisons?

Response: We acknowledge that the rating-scale floor can affect κ_E. Defective answers have lower initial scores and are more likely to reach 1 under the same latent penalty, resulting in a smaller observed penalty. A positive interaction on the 1–5 scale alone therefore cannot establish that verification endorsement makes judges more lenient. To examine the scope of this explanation, we checked the native reward scores in the original manuscript, which were not mapped to the 1–5 scale:

Reward modelInput settingQuestionsκ_E95% confidence interval
SkyworkAnswer-only (AO)620+0.1509[+0.0943, +0.2087]
SkyworkSource-aware (SA)494+0.1449[+0.1046, +0.1861]
AllenAIAnswer-only (AO)620−0.0071[−0.0264, +0.0131]
AllenAISource-aware (SA)494−0.0288[−0.0493, −0.0082]

Skywork's positive interaction persists in native scores without truncation at 1, indicating that this differential response does not necessarily depend on that rating-scale floor. AllenAI does not exhibit the same pattern. Here, the absolute scores of both types of answers decrease: positive κ_E means that the defect branch receives a smaller penalty, not that incorrect answers obtain an absolute reward gain.

We also selected 100 questions in two batches from the repaired candidate data and compared defect detection for identical defective answer bodies with and without verification endorsement. Among 700 model–question pairs, 26 changed from non-detection to detection, and 21 changed in the opposite direction, suggesting that verification endorsement may affect defect-detection judgments for some samples. Overall, non-detections decreased from 50 to 45, a change of −0.71 percentage points in the non-detection rate (95% confidence interval from question-cluster resampling: [−2.43, +1.14]). This supplementary experiment shows bidirectional changes in individual judgments, providing a behavioral observation complementary to the scoring experiment; a stable change in the overall detection rate has not been established.

AuthorsOpenReview ↗

Response to Reviewer SZXP (3)

Question 4: Appendix B.2 states that the naturalization interaction was identified after observing outcomes and that the endorsement interaction is exploratory. Can the authors validate these findings on an independent held-out set?

Response: We froze 50 questions in advance for held-out validation from a candidate pool that had undergone cleaning, repair, and semantic screening, with overlaps and near-duplicates of previously evaluated samples excluded. The sample contains 14 ALCE questions, 12 ExpertQA questions, and 24 RAGTruth questions. The sample, evaluation protocol, and analysis rules were fixed before this evaluation; questions were not replaced and the sample was not expanded in response to the results. Both reward models completed all 1200 native-score evaluations. The table reports the interactions and 95% confidence intervals obtained by source-stratified resampling at the question level:

Reward model and input settingκ_N [95% confidence interval]κ_E [95% confidence interval]
Skywork · answer-only+0.5244 [+0.0937, +1.0779]+0.3969 [+0.2472, +0.5605]
Skywork · source-aware+0.5984 [+0.2087, +1.0382]+0.4325 [+0.2700, +0.6180]
AllenAI · answer-only+0.1147 [−0.0150, +0.2661]−0.0048 [−0.0722, +0.0636]
AllenAI · source-aware+0.1766 [+0.0202, +0.3601]+0.0965 [+0.0273, +0.1683]

Skywork reproduces a positive E-track interaction under both input settings, while AllenAI shows a positive interaction under the source-aware setting. This replication in native reward scores indicates that Skywork's differential E-track response does not depend on artificial truncation to a 1–5 scale. The mean absolute scores of both branches decrease, so the positive interaction reflects a relatively smaller penalty for the defect branch.

The seven judges returned 2022/2100 valid outputs, and their κ_E point estimates are positive in their respective available pairs. The table reports the numbers of complete questions for the N/E tracks and the results aggregated across scoring rubrics. Under the prespecified rules, 95% confidence intervals are computed only for entries covering all 50 questions; descriptive means are reported for the remaining entries.

Judge modelQuestions N/Eκ_N [95% confidence interval]κ_E [95% confidence interval]
GPT-5.450/47+0.056 [−0.014, +0.135]+0.514 (CI not computed)
Claude Opus 4.850/50+0.065 [−0.015, +0.165]+0.102 [+0.028, +0.180]
Gemini 3.1 Pro50/50+0.071 [−0.030, +0.187]+0.690 [+0.599, +0.782]
DeepSeek V4 Pro49/49−0.010 (CI not computed)+0.329 (CI not computed)
GLM-5.250/48+0.050 [−0.019, +0.126]+0.261 (CI not computed)
Grok-4.339/29+0.541 (CI not computed)+0.771 (CI not computed)
Qwen 3.7 Max50/50+0.025 [−0.081, +0.130]+0.777 [+0.646, +0.910]

The held-out validation further supports the paper's finding of differential scoring responses in the E track. Claude, Gemini, and Qwen, each covering all 50 questions, have κ_E values of +0.102, +0.690, and +0.777, respectively, with 95% confidence intervals that exclude 0. Together with Skywork's native reward results under both input settings, this evidence supports different scoring responses to verification endorsement for the reference and defect branches and shows that the E-track pattern extends to held-out samples. The supported finding here is the relative scoring pattern in the E track. The direction of the N-track result in the new sample differs from that in the original evaluation and is reported separately.

AuthorsOpenReview ↗

Follow-up on Rebuttal

Dear Reviewer,

Thank you for your time and effort in reviewing our paper.

We have provided detailed responses to your concerns, including additional experimental evidence.

As the discussion period is approaching its end, we would greatly appreciate it if you could take a moment to review our responses.

Thank you again for your time and consideration.

Best regards, The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.