- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper introduces EvoCUE, a framework that improves agents by learning reusable edits to an explicit state-machine controller while keeping the model and tools fixed. It proposes edits from execution traces and residual goals, compares parent and edited continuations from shared checkpoints, and confirms updates on separate training tasks before inheritance. Its principal AppWorld result recovers an omitted task-completion convention, increasing Test-Normal/Test-Challenge success from 13.7%/6.7% to 79.2%/70.0%. A PAST-Bench pilot reports improved transfer of organizational requirements, with task-execution scores increasing from 0.608 to 0.772.
Critical strengths
-
It designs a coherent and inspectable procedure for persistent agent improvement. Matching the environment state, execution history, controller state, and remaining budget before comparing continuations is a sensible way to remove variation originating before an edit acts. Separate whole-program confirmation and records of rejected candidates provide a useful audit trail (§§3.3–3.4).
-
It shows substantial transfer within the AppWorld setting. The learned instruction improves hundreds of frozen test executions, with 110 improvements and no regressions on Test-Normal (§4.2). Additional runs without the historical hypotheses recover the same convention (Appendix E.2). This supports a reproducible ability to acquire and reuse missing procedural knowledge from public execution evidence.
Critical weaknesses
-
The experiments insufficiently isolate the distinctive contribution of localized control and checkpoint testing. In 4.4, the first-entry instruction is absent from subsequent prompts by construction; the comparison therefore primarily establishes the importance of continued instruction availability. Providing the same learned text at every step also solves 14/16 AppWorld tasks, and achieves 0.767 versus 0.772 on PAST-Bench. Both successful controllers use unconditional instruction attachments without new skills. A controlled comparison against task-start evaluation is also missing. This matters because AutoGuide already selects guidance according to the current state, while Self-Harness already validates persistent harness edits and reports learning the AppWorld action-versus-question completion distinction. The paper cites these predecessors, but the additional benefit of its particular mechanism needs stronger evidence. arxiv.org
-
Evidence for the complete learning procedure beyond the restricted AppWorld setup is weak. Recovering intentionally omitted onboarding information is a valid experiment, but the independent supporting evaluation is limited. The 16-task PAST pilot was previously inspected during development). On unseen supported families, the complete EvoCUE procedure accepts no update: 0.305->0.332 reflects rerunning the unchanged controller. The stronger 0.345>0.729 result comes from installing all distilled knowledge despite rejection by the confirmation stage (Table 2). None of the six additional domains yields an accepted update. These disclosed results leave broader generalization of the proposed selection-and-inheritance procedure insufficiently established.
Questions for authors
-
GEPA supports updates to individual module prompts. Which controller locations could the adapted GEPA baseline modify, and were its learned instructions available at subsequent action-generation calls? This would help determine whether the comparison isolates the learning algorithm or also changes the available editing interface. arxiv.org
-
With the proposer, public evidence, candidate space, and learning budget fixed, how much do checkpoint trials improve learning outcomes or cost over task-start trials and a learner that maintains instructions at every step? Does the advantage persist when the official AppWorld onboarding instructions are included?
-
Why does rule fitting exclude scored, applied trials without an observed behavior change (Appendix A.3)? For the accepted edit, this changes the estimated effect from 5/8 to 5/7. Are fitted rules and acceptance decisions stable when all scored, reached trials are retained?
3 comments
Response to Reviewer JYum (2/2): generalization, GEPA's editing interface, and rule fitting with all trials
W2. Evidence beyond the restricted AppWorld setup.
- PAST, unseen families, complete procedure (Section 4.5, Table 3, Appendix G.3).
- Every learning run now accepts updates.
- On held-out official episodes never run during learning, the task-execution score rises by up to +0.446 [0.247, 0.633], from 0.282 to 0.727, with six tasks improved and none declined.
- The gated procedure now reaches the level of installing all distilled knowledge (0.729).
- τ²-bench (Section 4.6, Appendix G.4). We reran it with the official 200-step limit, with GEPA and ACE alongside. In telecom, EvoCUE accepts two updates and is the only method that improves, reaching pass¹ = pass² = 1.000.
- AppWorld with the official instructions.
- EvoCUE learned from exceeds with all 17 official onboarding instructions by 6.5 / 14.2 points, and ACE by +18.8 / +24.5.
- Learning from gives +7.3 [3.1, 11.9] on Test-Challenge.
Q1. GEPA's editing interface.
- What GEPA edits. In our adaptation, GEPA optimizes the agent instruction. That instruction is included in every action-generation call, so its learned content is available at every subsequent step. AWM's memory enters the same way (Appendix D). GEPA therefore edits one instruction that acts at all locations.
- Isolating the interface. The new experiments hold the learner fixed and vary only the interface.
- The location interface wins by +9.9 / +13.8 (Table 2 f).
- With ACE's text fixed, it wins by up to +11.3 (Table 2 g–h).
- The interface is therefore a measured part of EvoCUE's advantage, in addition to the learning algorithm.
Q3. Rule fitting and trials without a behavior change.
- Why such trials are excluded. A trial in which the edit is applied but behavior does not change carries no information about the edit's effect at that state. Including it dilutes the estimate toward zero.
- Refit with every scored, reached trial (Appendix G.5, referenced from A.3). We refit on all 22 learning records: 164 rounds, 486 candidates and 19,619 trials, of which 1,358 are newly included.
- The submission's accepted edit. Its estimate moves from 5/7 to 5/8, its rule stays empty, and the learned controller is unchanged.
- Confirmation decisions. None of the 164 rounds changes which candidate reaches confirmation.
- Accepted edits. 31 of 33 keep an identical or runtime-equivalent rule.
- Why only checkpoint trials are affected. All newly included trials are checkpoint trials, because an edit tested from the task start almost always changes behavior.
We hope these results resolve the reviewer's questions, and we would be grateful if the reviewer would take them into account in the final assessment.
Follow-up on the new experiments
We thank the reviewer again for the constructive review. Our replies above report the learner-fixed experiments requested in Q2: learned locations are 9.9 / 13.8 points better than every-step availability, and checkpoint trials accept no neutral or harmful edit. We would be glad to hear whether these results address the reviewer's questions, and we are happy to answer anything that remains unclear.
Response to Reviewer JYum (1/2): the requested learner-fixed comparison, with and without the official onboarding
We thank the reviewer for the balanced assessment and the concrete questions. Q2 describes exactly the design we ran. All new results are in the revised PDF (new Table 2, Sections 4.2–4.6, new Appendix G).
Protocol of all new runs.
W1 / Q2. Checkpoint vs. task-start trials, and learned locations vs. every-step instructions, with proposer, evidence, candidate space and budget fixed.
From C0, one learner varies only the location or only the trial type (Table 2 f, i):
Relation to AutoGuide and Self-Harness.