- Rating
- 1(−1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper examines false rejections of valid answers under reference-based evaluation and their local optimization effects when these judgments are used as rewards. Evidence-based verification across five search benchmarks yields a 2.12-percentage-point increase in acceptance rate, while source-based attribution distinguishes strict reference omissions from other grading errors. The authors then hold the initial model and trajectory groups fixed, correct a target reward, and measure how a single parameter update changes the likelihood of valid answers and useful search queries. Additional comparisons examine response content, group composition, and evaluation context.
Critical strengths
-
The distinctions between grading errors are practically useful. The paper carefully separates reference omissions, defective references, equivalent expressions, and interpretation differences, while assessing task-answer and full-response validity separately. This helps identify what requires correction rather than attributing every disagreement to incomplete answer coverage.
-
The paired-update design provides a reasonable basis for local analysis. Holding the model and trajectories fixed while changing only the target reward helps isolate the intervention’s total effect. The authors distinguish relative improvement from recovery above the initial likelihood and report inconsistent correction effects rather than interpreting local measurements as evidence of long-term training gains.
Critical weaknesses
1. The substantive connection between reference omissions and the gradient analysis is insufficiently established. The optimization analysis primarily concerns binary reward corrections on fixed trajectories, and its central derivations apply equally to false rejections arising from other causes. Although the authors report some results by error category, it remains unclear whether reference omissions exhibit systematic structure in the affected answers, search paths, or gradient effects beyond general reward errors. Both parts have value individually, but the additional insight obtained by connecting them is not yet sufficiently clear.
2. The likelihood changes are observed, but the gradients’ specific contributions and functional effects remain unclear. The paper defines alignment between trajectory gradients and the evaluated target, yet mainly reports the resulting likelihood changes. This makes it difficult to determine whether an effect comes from the target trajectory, companion reweighting, or interference between different content segments. Moreover, changes in a particular query string’s likelihood do not establish changes in search-intent preference, and the existing next-action check finds no consistent selection benefit. The optimization section therefore remains closer to a demonstration of local phenomena than a sufficiently validated mechanism analysis.
Questions for authors
1. Error specificity. What additional optimization insight does reference omission provide beyond general false-rejection effects? A comparison with other error types or randomly introduced false rejections, under matched group sizes and reward counts, would help clarify this.
2. Gradient contribution. Could you identify which gradient components drive the observed suppression or improvement—for example, the target versus companion trajectories, or search versus final-answer tokens? Comparing first-order predictions with actual likelihood changes would also help validate the mechanism.
3. Functional effects. Beyond changes in particular strings’ likelihoods, do the updates affect preferences for valid answers or useful search intents? Candidate-ranking tests or an extension of the next-action check in Appendix F.3 could help establish this.
I downgrade my scores due to the lack of response from the authors.