- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces AutoPEER, an automatic prompt optimization framework designed for LLM agents operating in long-horizon, stateful tool environments. Rather than rewriting entire prompts or endlessly accumulating experience in a sprawling playbook, AutoPEER couples Issue-level temporal credit assignment with three-stage progressive behavioral verification:
Critical strengths
A central bottleneck in agentic prompt evolution is that trajectory-level feedback is noisy and full-episode validation is computationally expensive, while unconstrained playbook accumulation (like ACE) leads to severe context bloat. AutoPEER addresses both challenges through a cohesive pipeline. By forcing the optimizer to ground prompt edits in specific trajectory timestamps (replay targets) and verifying them via Merge and Rolling across recent batches, the framework effectively filters out spurious local edits (with Full-Episode Validation rejecting ~42% of candidates that pass Fast Replay, and Rolling blocking destructive patch merges) while adding 75.7%–94.5% fewer characters than ACE. This provides a highly practical, traceable paradigm for updating agent instructions without weight updates.
Critical weaknesses
-
Although Fast Replay is presented as a principal component of the three-stage verification pipeline, the ablation results in Tables 4 and 6 indicate that excluding this module (w/o Fast Replay) achieves identical final TGC on AppWorld Normal (86.31%) and Challenge (76.26%). Furthermore, its removal decreases the number of candidates forwarded to Full-Episode Validation by only 11.8% (from 127 to 112), corresponding to a savings of merely 45 validation episodes over 30 generations. Notably, Fast Replay resets the environment and reruns recorded code or tool prefixes up to turn , adding simulator overhead and requiring deterministic, replayable states. The acceptance rule (“a patch passes if the action changes and executes successfully”) may also reject valid patches whose effects appear later (e.g., at turn ) or that prevent later errors without changing the action at turn .
-
The initial baseline prompts were overly detailed and lacked analysis of the stability of the training order and ground truth feedback. As seen in Appendix E.1, the initial system prompts were not simple general instructions, but were manually divided into eight clear sections (PE-01 to PE-08), with very specific AppWorld rules pre-written (such as checking the password in apis.supervisor, using page_index to navigate to the bottom of the pagination interface, treating the file system as an application rather than a system file, and specifying when to pass the answer parameter to complete_task). This is equivalent to pre-setting a well-structured framework for local patching. Whether this local patching method would still be effective with a very rudimentary initial prompt remains to be verified. In addition, the training tasks were only run once in a fixed order, without testing the variance caused by shuffling the batch order. Moreover, the effect of providing or not providing a reference ground truth (GT) was unstable. For example, on the DeepSeek Challenge testset, the task completion rate without a ground truth (78.18%) was actually higher than that with a ground truth (76.26%).
Questions for authors
- Since AutoPEER visits each training scenario once in a fixed sequential order and only checks regressions on the latest batches (up to 9 tasks), how sensitive is the final evolved prompt to the ordering of the training batches? Have you observed cases where an update accepted at generation causes regressions on tasks from generations , and how does performance/cost scale if is increased or sampled via stratified replay across earlier scenarios?
- In Table 6, Fast Replay filters out only 16 out of 128 valid candidates (~12.5%) before Full-Episode Validation, and w/o Fast Replay achieves identical TGC on both Normal and Challenge (Table 4) (though SGC on Challenge is 56.83% vs. 61.87%). What is the exact wall-clock time and token/execution cost of running Fast Replay (3 paired trials up to 3 replay targets per Issue) compared to the 45 full-validation episodes it saves?
- On AppWorld Challenge with DeepSeek-V4-Flash, AutoPEER without GT achieves higher TGC (78.18%) than AutoPEER+GT (76.26%), despite AutoPEER+GT achieving higher SGC (61.87% vs. 59.71%) and higher Normal TGC/SGC. Could you elaborate on what types of patches GT induced that lowered task completion on Challenge? Does exposure to reference solution code cause the Patch Generator to propose overly specific rules that overfit to training scenario structures?
Thank you for your feedback. We have responded to your questions above and would appreciate hearing your thoughts soon.