- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes DUET, a framework that jointly optimises the prompts (optionally, skills) of a solver agent and a grader agent without needing reference answers during training updates. The paper shows that DUET achieves higher solver scores than SkillOpt and GEPA, as well as higher grader pairwise agreement than an initial grader on constructed pair sets.
Critical strengths
-
The results are carefully reported with proper seeds, standard deviations, ablations of each design element, and hyperparameter sensitivity analyses. The paper also includes the full updater prompts and learned prompts, which is appreciated.
-
The problem is well motivated, and the reference-free setting where task instructions and environments exist but references and rubrics don't is realistic for open-ended agent tasks.
Critical weaknesses
- Much of the gap to the baselines seems to come from the optimiser rather than from co-evolving the grader. For example DUET(G⋆) which fixes the oracle grader and only optimises the solver already beats SkillOpt(G⋆) and GEPA(G⋆) on τ²-Bench and GDPval.
- In Appendix C.14, three of the four learned grader prompts contain explicit instructions about emitting a verdict that can be parsed reliably by the evaluation harness. Similarly the GDPval prompt notes that grades were silently discarded by the parser and recorded as a default middle score. This makes me wonder how much of the evidence for grader improvement is attributed to better compliance with the evaluation harness's way of parsing outputs?
- The update agent is instructed to search the web (Section 4.3, Appendix B.2), with no domain restrictions described, while all four benchmarks publicly release reference answers, gold deliverables, or grading rubrics. This is cause for concern to me, since the updater (GitHub Copilot CLI with Claude Opus 4.8) could plausibly have encountered this material without anyone intending it (see my question below).
Questions for authors
-
Could you report optimisation training costs (tokens or dollars, including updater, grader and validation calls) for DUET versus the baselines?
-
Could the authors confirm whether the benchmarks' domains were blocked, and whether search logs were audited?
8 comments
Responses to Reviewer MkzJ (Part 2)
W2: How much of the evidence for grader improvement is attributed to better compliance with the evaluation harness's way of parsing outputs?
We thank the reviewer for raising this concern. The parsing-related instructions were learned to address output-format issues encountered during optimization. To investigate this, we audited grader parsing and conducted additional evaluations. On GDPval, even when restricting the evaluation to pairs for which both graders produced valid scores, the learned grader achieves higher pairwise agreement than the initial grader (77.8% vs. 60.7%). Furthermore, in our additional cross-checkpoint evaluation on τ²-Bench in our response to W1, all 1,080 grader evaluations were successfully parsed, and overall AUROC still improves from 0.707 to 0.771. These results provide evidence that grader improvements are not solely due to better output-format compliance.
Q1: Optimization cost compared with baselines.
DUET and the baselines (SkillOpt and GEPA) all require solver rollouts, grading or reward evaluation, and solver optimization. DUET additionally updates the grader during optimization. In a 30-round τ²-Bench run, these two steps accounted for 8.2% of the total optimization cost. This suggests that DUET achieves better solver performance than the baselines with relatively modest additional optimization cost from grader evolution.
Furthermore, the baseline methods require a predefined evaluator, which may involve manually designing task-specific grading criteria or collecting reference answers. DUET can learn its grader starting from a minimal prompt, reducing the need for manual evaluator design while achieving competitive solver performance.
We have included this analysis in Appendix C.10 of the updated version.
W3 and Q2: Benchmark contamination through web search.
We thank the reviewer for raising this concern. We would like to clarify how web access is controlled in DUET.
Who can search: In DUET, only the updaters can use web search to research general improvement strategies. The solver and grader agents do not have web-search tools, except on the Finance Agent Benchmark, where web search is part of the benchmark environment, as this benchmark contains web research tasks.
What the updater sees: The updaters see only training-split tasks and their trajectories; held-out test tasks, outputs and scores never enter their workspace, so the updater has no test-task content to search for.
What the updater searches for: The scope of the updater's web searches focuses on general strategies for improving agent performance instead of benchmark-specific answers. For example, the solver updater is instructed to "web search current practice (agent design, tool-use, planning, self-verification)" when proposing improvements.
What the updater may write: Its output must be a general prompt, and rules that "hard-code a sheet name, cell address, or value" are considered overfitting and are not allowed. So even a retrieved answer would have nowhere legitimate to go.
We also audited the updater's web-search logs and found no benchmark reference answers were retrieved during optimization. For example, on the Finance Agent Benchmark, we found no benchmark repository pages or grading-rubric assertions in the retrieved content.
We thank the reviewer again for the helpful comments. If our responses have addressed your concerns, we would appreciate it if you could consider raising your rating. We would also be happy to address any further questions you may have.
Dear authors of Submission12660,
Thank you for your rebuttal and for including revisions to the paper following the review(s). You have addressed my question regarding the web search tool.
To clarify my original question regarding costs: I was hoping for the total optimisation cost of DUET compared with SkillOpt and GEPA (e.g, total tokens or dollars per run, including updater, grader, and validation calls). Since DUET runs validation every round and uses a tool-using updater with web search, the absolute totals would help readers judge the comparison.
Regarding the controlled feedback comparison (Section 5.3, Table 6):
-
Please report mean & SD over the three repeats and state the number of runs in the caption. On -Bench, the gap is 0.8 where Table 1 reports SDs of 2.1-6.4 on this benchmark. It's difficult to interpret this as a difference without variance estimates.
-
Could you describe the setup more fully? For example how many update steps were applied to , how many training trajectories were re-graded, which DUET run was taken from, and what is different across the repeats?
-
The -guided result on -Bench (88.1%) is higher than the fixed-grader ablation in Table 14 (83.0%) and SkillOpt/GEPA in Table 1 (84.7% / 83.3%). Could you explain this difference? If this is already reached by updates guided by the initial grader, it would suggest to me that much of the gain comes from the updater rather than from grader co-evolution.
Also: the initial-solver results in Table 6 differ from in Table 1. Are these separate evaluation runs?
Minor presentation nit: could you please report the standard deviations, as well as the number of runs consistently throughout the revisions? For example Table 10 is lacking the standard deviations that were given in Table 1.
Responses to Reviewer MkzJ (Part 1)
We thank the reviewer for the careful reading and thoughtful comments. We address the reviewer's questions in the following and have revised the paper accordingly. The changes are marked in blue in our revision. We hope the responses below address the reviewer's concerns.
W1: Much of the gap to the baselines seems to come from the optimiser rather than from co-evolving the grader.
We thank the reviewer for this insightful comment. To isolate the contribution of co-evolution, we conducted ablation studies and additional controlled experiments.
Co-evolution ablation. In Figure 3(a), we compare DUET with three ablations on Finance Agent Benchmark: (1) optimizing only the solver with a fixed grader, (2) optimizing only the grader with a fixed solver, and (3) sequential optimization, where the grader is optimized first and then frozen during solver optimization. DUET achieves the strongest overall performance, supporting the benefit of jointly optimizing the solver and grader within the same framework. We also extend this ablation to τ²-Bench in Appendix C.13 and observe similar results.
Cross-checkpoint grader evaluation. To assess whether grader quality improves as the solver evolves, we selected initial, intermediate, and final solver and grader checkpoints from the same DUET run, denoted as A0, Amid, and AT for the solvers, and G0, Gmid, and GT for the graders. Then we used each solver to perform the same held-out tasks, and each grader to evaluate the resulting outputs. We used the benchmark's independent success checks to measure solver performance and grader AUROC, which reflects the grader's ability to distinguish successful from failed executions.
As the solver evolved, its success rate independently measured by the benchmark improved from 80.0% to 88.3% to 89.2%. The final grader achieved higher AUROC than the initial grader across all solver stages, with overall AUROC improving from 0.707 to 0.771. These results provide additional evidence that grader evolution improves evaluation quality as solver behavior changes.
Controlled comparison of grader feedback. To assess whether the learned grader provides more useful feedback for solver optimization, we conducted controlled experiments on SpreadsheetBench and τ²-Bench. Starting from the same initial solver checkpoint A0, we used G0 and GT to independently evaluate the same training trajectories and generate scores and critiques. We then applied the same updater to update the solver, repeating each condition three times. The resulting solvers were evaluated on the same held-out tasks using the benchmark's reference evaluator.
Our results show that updates guided by the learned grader achieve higher solver performance on both benchmarks. Since the updater is fixed, these results provide additional evidence that improved grader feedback contributes to solver optimization.
Motivation for SkillOpt (G⋆) and GEPA (G⋆). In our experiments, we include SkillOpt (G⋆) and GEPA (G⋆) as stronger baselines by allowing both methods to use the benchmark's reference evaluator during solver optimization. Here, G⋆ denotes the reference evaluator provided by the benchmark authors, which may rely on reference answers and manually designed task-specific grading criteria. This provides them with a strong, task-specific evaluation signal that DUET does not require. Despite optimizing from a minimal grader without ground-truth supervision, DUET outperforms both baselines across all four benchmarks, even when they use G⋆.
We have included the cross-checkpoint grader evaluation in Section 5.1 and the controlled comparison of grader feedback in Section 5.3 of the updated version.