- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper tries to answer the question can AI research agents get higher scores on science benchmarks without doing better science? The paper calls this “evaluation hacking”.
The authors test this by giving the AI the same tasks twice, once asking the AI to be responsible (focus on accuracy, evidence, admit uncertainties) and once asking the AI to be 'score-maximizing'.
Then they grade the results with the benchmarks' scores and use an AI checker to verify the evidence behind the AI's answers.
The benchmarks are ResearchClawBench, MLR-Bench, and SciAgentGym.
Their main results are: 'score-maximizing' increases the benchmark's grade but lowers the support; the pattern differs by model; the models notice the metric but rarely plan to cheat; a support-aware scoring rule helps in two of three benchmarks; and a model whose refusal ability was removed doesn’t chase the score more.
Critical strengths
The idea is intuitive and easy to follow, and the underlying question the authors are trying to answer is important to ask.
Critical weaknesses
- The writing is hard to follow due to: repetitive words; constant references to earlier versions that the reader never sees (“Earlier analyses… are kept separate”, “replace the earlier unreproduced value of 0.0014”); numbers printed with too many digits (0.8135593220, 0.9751007825); the sudden introduction of concepts that were not defined earlier (“RSI evaluations”, “utilize memorized data or look for easy answers online”, ε_S and ε_Q, “IRL science reward”, etc.); and several invented terms such as “fixed-science vulnerability,” “score pressure,” “frozen-path audit,” “strict unsupported-success endpoint” and “harmful switch.” Throughout the paper, the text also reads defensively, with many disclaimers following the results, making the reader wonder why the authors raise them.
- The prompts of the AI checker are missing, so a reader cannot determine whether the main finding is real or comes from biased prompting.
- The authors test a multi-model system throughout the paper, but they keep changing the models behind the multi-model system, the graders, and the test types across benchmarks. The lack of consistency makes their evidence weaker.
- There are existing related works that the authors do not discuss, such as: a. Hidden Pitfalls of AI Scientist Systems. It finds AI-scientist systems misusing metrics and picking results after the fact. b. LMs Learn to Mislead Humans via RLHF (ICLR 2025). Models become more convincing without becoming more correct. c. Reward Hacking Benchmark (RHB) (May 2026). Agents skip verification steps in tool use, like the SciAgentGym shortcuts. RHB also links reinforcement-learning post-training to exploits, which is the training claim this paper makes but cannot show.
- Although the paper contains many studies, the scope of most of them is small: ResearchClawBench: 20 tasks, or 39 in the choice study MLR-Bench: 10 tasks SciAgentGym: 13 tasks ResearchClawBench and MLR-Bench mainly use one agent model each.
- The abstract blames training, but only the instruction is tested, and the body admits this five times.
- The authors say their human audits don’t validate the checker’s quality scores (l.755).
Questions for authors
- Is it possible to share the prompts you used for the AI checkers?
- Is it possible to rerun the experiments with a consistent model configuration?
- Is it possible to adjust the scope by going deeper into each study and narrowing the main claims, while discussing the existing literature more thoroughly?
- Is it possible to define the terms that are uncommon in this field before using them?
4 comments
Dear reviewer,
This is just a gentle reminder to kindly read our rebuttals. We have roughly 4 days left. If you have any remaining concerns please let us know.
Best, The Authors
Respected Reviewer,
Please let us know if your comments are addressed or not. We just have 1.5 days left we would be very grateful if you can let us know what you think of our rebuttals.
Best, The authors
Respected Reviewer,
Please let us know if you have any additional questions. We have less than 48 hours.
Best,
The Authors
Dear Reviewer,
This is just a gentle reminder that we have 8 days left, please kindly let us know if you have any remaining concerns.
Best, The Authors