- Rating
- 2
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper audits reward hacking patterns across several agent benchmarks and proposes a weak-to-strong verifier hardening method built on an isolated sandbox and a hacker-fixer loop, in which weaker models repair the verifier against stronger agents. On TerminalBench, KernelBench and SETA, attack success rates drop substantially while benign solve rates remain largely unchanged, and two ablation groups separate the contributions of hardening capability and benign validity.
Critical strengths
- The problem is clearly posed and targets reward hacking in benchmarks, which has clear practical relevance.
- The ablations are thorough, and the multiple comparisons both identify the mechanisms behind reward hacking and support comparisons across method components.
Critical weaknesses
- Evaluation relies on a single judge backend from the GPT family, which may introduce same-family bias, and no agreement rate between this judge and human annotation is reported.
- Patch acceptance depends on whether the current model can solve the task, and hard-task settings are discussed insufficiently.
Questions for authors
- Could the authors report ASR re-judged by additional judge models, so as to show how sensitive the conclusions are to the choice of judge?
- Could the authors walk through the hardening process on a few hard tasks, showing how a patch is accepted or rejected when the weaker model cannot solve the task?
- Benign pass rates drop after hardening (on SETA, Gemini falls from 92.5% to 84.5%). Could the authors explain where these drops come from?