- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper studied whether agents that improve their own harness become better at solving tasks or merely better at satisfying their evaluator. With model weights fixed, it separates feedback used to select scaffold revisions from target performance measured on held out tasks, comparing aligned evaluators with incomplete proxies in counterpoint music repair and SQL repair. Across generations, proxy feedback increases visible scores while reducing target performance.
Critical strengths
The paper show that evaluator choice can change the direction of harness improvement, which is informative to the current self-improvement studies. Within each domain, the starting harness, model, search procedure, and resource limits remain fixed, while selection uses either aligned feedback or an incomplete proxy. Independent target evaluation reveals substantial degradation under proxy feedback in both music and SQL repair.
The study also has complementary controls and program audits, which makes it more solid. Fixed solver best of 5 selection produces less target degradation than harness evolution under the same inference call cap, and withholding diagnostic reports reproduces the negative direction. Source code audits further document evaluator specific prompting, scoring, and selection procedures in retained scaffolds.
Critical weaknesses
The conceptual novelty is kinda limited. The central finding that optimizing a misaligned proxy can degrade true performance is already well established in reward hacking literature. Extending this observation to harness evolution provides a new setting, but the paper offers limited additional mechanistic insights or solutions.
Also, the experiment scope is narrow. Both counterpoint and SQL repair use synthetic tasks with deliberately incomplete proxy evaluators, making the observed failures somewhat predictable. The small evaluation sets and lack of realistic agent benchmarks further limit the generalizability of the conclusions.
And attribution to scaffold evolution does not seem to be sufficiently established, as the comparison with best of 5 selection lacks a direct statistical test and does not control for outer loop computation.
Questions for authors
See weaknesses