- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes PairEvolve, a framework for reducing task-reward dependence in self-evolving agents. For prompt evolution, PairEvolve removes task scores and gold answers from mutation feedback and uses pairwise LLM judgments to decide whether to accept a revised prompt. Two variants are considered: PairEvolve-Val retains validation-based parent selection, while PairEvolve-Elo uses Elo ratings derived from pairwise comparisons. Experiments on four prompt optimization tasks show improvements over the initial prompts in several settings, although performance relative to full-reward evolution varies across tasks. The paper further investigates validator biases, acceptance rules, and extensions to program evolution.
Critical strengths
-
The paper provides a useful decomposition of reward dependence into mutation, acceptance, and parent selection. The distinction between PairEvolve-Val and PairEvolve-Elo is clearly motivated, and the experiments demonstrate that useful prompts can be obtained without task-reward feedback at every evolution step.
-
The validator analysis is a valuable part of the paper. In particular, the investigation of position bias and tie handling reveals an important failure mode: pairwise comparisons can provide little effective selection pressure when ambiguous judgments are frequently accepted.
Critical weaknesses
-
The benefit of pairwise acceptance itself is not convincingly isolated. In Table 8, the no-validator baseline performs comparably to or numerically better than symmetric pairwise acceptance on both tasks. Although this experiment uses a different configuration from the main study, it raises the question of whether the gains in Table 1 primarily come from reward-blind mutation and downstream candidate selection rather than the proposed validator. A matched control for the main configurations is missing.
-
The empirical evidence is somewhat limited by the evaluation protocol. The test sets contain only 30--50 examples, and several tasks use modified or permissive metrics, including partial constraint satisfaction on IFBench and substring-based label extraction on HoVer. Together with the substantial variability in some results, this makes the robustness of the reported gains uncertain. The program-evolution results are also based on a single run per configuration.
Questions for authors
-
Could the authors include a matched no-validator or unconditional-acceptance baseline for the primary experiments? This would help establish whether pairwise judgments provide benefits beyond reward-blind mutation and downstream selection.
-
How sensitive are the main results to stricter evaluation metrics, particularly for IFBench and HoVer? Would the relative rankings remain similar?
-
Since task evaluators still execute before their scores are masked (Appendix A.2), could the authors quantify the reduction in labeled feedback requirements and the additional inference cost introduced by pairwise judging? This would clarify the practical benefit of the proposed reward-substitution strategy.
Response to Reviewer RHrp: Breadth, reward access and scoring verification
Useful evolution with less task-reward feedback
PairEvolve shows that useful evolution does not require task rewards at every search decision.
Both PairEvolve variants compare the parent and revised prompt’s answers to decide whether to accept the revision. After search, all methods choose the highest-validation-score prompt and test it on separate questions.
For example, GPT-5.4-mini HotpotQA Val/Elo reach 84.0/79.8 test F1, versus 29.2 initially and 77.1 with full reward. Both improve over the initial prompt in all three seeds; Val exceeds full reward in every seed.
1. W1 / Q1 — What do comparisons add to search?
LLM comparisons provide informative feedback. Table 2a uses constructed pairs: a control answer and a flawed alternative. Evaluators see the task and answers, without references or flaw annotations. Pairwise compares answers together; pointwise scores them separately, favoring the higher score.
Agreement with the designated better answer (%) ± standard error; thinking enabled, same valid pairs, half credit for ties. Pairwise is 39.8–47.8 points above chance (50%).
GPT-5.4-mini HotpotQA Elo’s selected prompts first received full-validation evaluation at iterations 327, 278 and 97 of 350: useful revisions continue to emerge late in search.
2. W2 — How broad and repeatable is the evidence?
96 runs span four tasks and two models: HotpotQA, HoVer, IFBench and PUPA; Qwen3-8B/GPT-5.4-mini use five/three seeds per method/task, with 30–50 test questions per task.
Mini HotpotQA Val exceeds full reward in all three seeds: +10.65, +5.86 and +4.17 F1. On mini IFBench, Val matches full reward’s mean 90.4 and improves over the initial prompt in every seed under the reported metric.
Beyond GEPA, ADRS and ShinkaEvolve apply comparisons to scheduling, circle packing, signal processing and prime counting: one run per configuration, with code-execution checks. The evidence spans prompt and program evolution.
3. Q3 — What reward access and inference does this require?
Elo uses task scores only for final selection. All methods use 30 validation questions with reference answers to select the final prompt. Full reward and Val also use these scores to choose parents during search. We reduce reward-guided decisions, not validation-set size.
Evolution cost. Comparisons reuse generated answers; Elo reuses judgments for ratings. Appendix F records 267,680 task calls/783.7M tokens and 26,100 validator calls/32.2M tokens, pooled over two tasks/four configurations, not per-method costs. Failed-attempt usage is unknown. Evaluators run before score masking, so reduced reward feedback does not establish compute savings.
Test-time cost. Each method deploys its selected prompt directly: no extra judge calls. Prompt/answer lengths can still change token use.
4. W2 / Q2 — How sensitive are the results to evaluation rules?
HoVer: Val accuracy rises from 38.4% to 45.2% under semantic verdict evaluation. To isolate scoring effects, we applied both rules to the same new Qwen3-8B thinking-on answers from fixed historical prompts: all five seeds, 50 common questions, 250 answers per method. Two LLM interpreters, blinded to reference labels, extract the intended verdict for every method.
Accuracy (%). Semantic evaluation accepts equivalent verdict wording.
IFBench and PUPA: no observed fallback effect in the audited evaluations. We observed zero fallback/default events in 1,144 IFBench constraint checks and 480 new PUPA judgments. All 588/588 auditable IFBench validation scores reproduced; 480/480 PUPA combined scores were unchanged between numeric parsers. PUPA uses fixed historical Qwen prompts with thinking enabled on 30 common questions. Neither new PUPA nor HoVer answers reconstruct historical test outputs.
Strict all-constraints IFBench scoring preserves aggregate method ordering on common retained validation questions across five Qwen seeds. We will correct attribution to THU-KEG IFBench (Peng et al.). Fallback handling leaves the checked IFBench/PUPA scores unchanged.