acceptodds
Withdrawn from ICLR 2027

ProSEvo: Self-Evolving Agents Without a Stronger Teacher

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Can an agent improve from its own interactions without a stronger model to teach it? We study this question with a single locally hosted generative backbone used for experience collection, offline compilation, and subsequent agent generations. The agent must select self-generated experience carefully: a successful trajectory can still contain erroneous steps. ProSEvo uses environment outcomes, source-linked event labels, and deterministic checks to select what persists. Supported procedures become typed memory for the role that will use them; admitted original outputs become memory-free supervision for the role that produced them. Each updated student then collects experience for its successor, so both memory and parameters evolve across rounds. Under a unified same-backbone protocol on OfficeBench, AppWorld, and -bench Retail, ProSEvo raises macro task success from % to %, versus % for the strongest controlled comparator, % with memory alone, and % with parameter updates alone. It also leads the compared methods on each benchmark under this protocol. Fixed-checkpoint and round-wise controls distinguish the contributions of compiled memory and cumulative student-generated experience.

Primary area:
foundation or frontier models, including LLMs
Keywords:
LLM Agents, Self-Evolving Agents, Self-Distillation, Agent Memory, Tool Use, Procedural Knowledge, Continual Learning, Multi-Agent Systems

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_59ADOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces ProSEvo, a framework for the self-evolution of multi-agent systems within a single generative backbone lineage. The method assigns interaction experience to controller and executor roles and retains it through two mechanisms: typed procedural memory and role-specific parameter updates. A frozen same-backbone compiler annotates recorded trajectories, while deterministic validation checks their structure and source evidence. Admitted experience supports memory construction, whereas original positive role outputs from successful trajectories provide supervision for separate student adapters. Through successive rounds of experience collection, memory accumulation, and supervised fine-tuning, ProSEvo improves performance over the base system on OfficeBench, AppWorld, and τ-bench Retail.

Critical strengths
  1. The framework clearly distinguishes memory consumption from parameter supervision. Procedural knowledge is routed to the role that should use it, while training targets remain associated with the role that originally produced the corresponding output. Together with explicit source and execution checks, this design makes the flow of experience through the learning pipeline relatively transparent and provides a useful framework for studying cumulative agent improvement.
  2. The evaluation includes multiple benchmarks and informative interventions. The paper reports consistent improvements over the base system and examines memory components, parameter updates, and cumulative experience reuse. In particular, the fixed-checkpoint comparisons between compiled, raw, absent, and role-swapped memory provide useful evidence about how inference-time memory affects performance.
Critical weaknesses
  1. The benefit of the proposed training-data admission and supervision construction over straightforward successful-trajectory replay is insufficiently established. The paper provides useful memory interventions and carrier ablations. However, Raw-replay SFT is explicitly introduced in Appendix A.3 and Table 9 without corresponding results. This leaves an important gap in assessing the additional value of the training-side filtering and supervision construction. The raw-trajectory retrieval comparison addresses inference-time memory and does not resolve this training-side question.
  2. The evidence for reducing self-confirmation is indirect. The paper uses self-confirmation risk to motivate ProSEvo, but the experiments do not directly test whether the admission rules reduce incorrect interpretations of past experience. For example, a failure explanation can cite the correct event and still misidentify why the action failed. The validation statistics in Table 26 report structural and source-check failures, without measuring how often incorrect interpretations are accepted or reused in later rounds. The task results are encouraging, but I would be more cautious about attributing these gains to reduced self-confirmation.
Questions for authors
  1. What evidence beyond structural and provenance validation supports the claim that ProSEvo mitigates semantic self-confirmation?
  2. Could the authors report the Raw-replay SFT results defined in Table 9? Please specify the inference-time memory condition and which supervision-construction choices change in this comparison, so that its results can be interpreted under the stated matched experience and optimization budgets.
  3. What specifically differs between the final Full checkpoint evaluated without retrieval in Table 20 and the Parameter-only variant in Table 23?
Reviewer_jbFEOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

I think this paper is AI generated, or at largely automatically generateed without careful polish.

Examples

  • L48-49: "the collector, compiler, and deployed agent roles are instantiated from one base checkpoint, with role-specific student adapters updated across rounds." None of collector, compiler, deployed agent roles are defined or illustrated anywhere. I don't think a human writer communicates this way but just throwing in abstract terms.
  • L57-59: "Localized failures provide correction guidance through memory rather than making an unsuccessful rollout a positive SFT target." This sentence does not make much sense with SFT comeing from nowhere.
  • L82-84: I read it 3 times, but couldn't really comprehend what this means. What's "consuming role", "producing role" and "original positive outputs"?
  • Eq (1): pc and ex are hallucinated without definitions
  • L188-189: "Let B denote the locally hosted base checkpoint, and let G0 denote a frozen, harnessed copy used exclusively for bootstrap collection." - what is "base checkpoint"? "harnessed copy" is not something a human would write. ... I stop here.

Unless the authors can provide clear evidence this is not AI written - I recommend rejection.

Critical strengths

NA

Critical weaknesses

AI written paper

Questions for authors

Please provide evidence of this paper being human written.

Reviewer_d7FeOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

ProSEvo explores how a tool-using agent can improve from its own trajectories by updating both an external procedural memory and its internal model weights. The system is built on a fixed controller–executor architecture where each role has its own LoRA adapter, alongside a frozen compiler sharing the same base model. The compiler distills raw trajectories into structure, execution, and failure memories. Meanwhile, environment outcomes and an event-level admission filter decide which logged actions are clean enough to be used for SFT. A neat design choice here is decoupling who reads the memory from who gets trained on the action: a compiled memory might be routed to help a different role downstream, whereas SFT supervision strictly goes back to the role that originally generated that action. The authors test this setup over three rounds of offline updates on OfficeBench, AppWorld, and τ\tau-bench Retail. Macro-average test success climbs from 17.39% to 29.12%, beating both the memory-only and parameter-only baselines. They also include ablations looking at memory routing and the gains from accumulating experience across rounds.

Critical strengths
  • This framework explores an important problem in agent self-improvement: how to turn raw interaction experience into lasting capability without relying on a stronger external teacher. Its consistent gains across three benchmarks over three offline rounds show that this dual-channel setup effectively translates past trajectories into downstream improvement on unseen tasks.

  • The paper presents a practical recipe for combining memory and parameter updates. The role separation makes practical sense. External memory captures high-level workflows and failture warnings, while LoRA updates internalize successful action patterns. The carrier ablations further confirm the effectiveness of combining both update methods.

Critical weaknesses
  • While the system is well-engineered, its core components (saving procedural memory, filtering successful trajectories for SFT, and routing by role) are fairly intuitive. The paper doesn't clearly articulate what fundamental bottleneck this specific combination solves compared to prior state-of-the-art.

  • The related work section needs sharper positioning. While EvolveR and LEGOMem are cited, the conceptual boundary between ProSEvo and their experience/role-memory designs feels blurry. More importantly, the paper misses COVE [1], which already studies how to coordinate external memory and parameter updates (including knowledge routing). ProSEvo needs to explicitly justify what its source-grounded admission and role attribution add on top of COVE. Discussing WHALE [2] would also help contextualize the work within the broader trend of jointly updating weights and external text/harnesses. Right now, ProSEvo reads more like a solid system integration than a distinct methodological advance.

  • Showing that "memory + SFT" beats either alone is good, but it doesn't prove the necessity of ProSEvo's specific event-level admission and attribution rules over a simpler baseline (e.g., standard procedural memory + SFT on full successful trajectories). Similarly, swapping or shuffling memories only proves that corrupted routing hurts instead of the ineffectiveness of a simpler shared memory pool. Appendix A.3 mentions a "raw-replay SFT" control, which is the baseline needed here, but its numbers seem to be missing. Reporting this under matched budgets is essential to prove the value of event-level processing. Finally, the evaluation relies too heavily on aggregate success rates. Given the straightforward architecture, the paper would benefit from a deeper qualitative/quantitative analysis, such as what toxic actions the admission filter actually catches, or which task types benefit from memory vs. weight updates.

[1] Ji, Tianyun, et al. "Learning What to Remember and What to Internalize in LLM Self-Evolution via Adaptive Memory-Parameter Coordination." arXiv e-prints (2026): arXiv-2608.

[2] Kim, Haechan, et al. "WHALE: A simple recipe for joint harness-weight optimization." arXiv preprint arXiv:2609.00196 (2026).

Questions for authors
  • Table 9 describes a raw-replay SFT control, which I couldn't find in the results. How does full ProSEvo compare against pairing the same procedural memory with standard successful-trajectory SFT?

  • Positioning against COVE and prior work: How do you differentiate ProSEvo from existing hybrid approaches, particularly COVE’s memory–parameter coordination? What specific failure mode does your role-and-carrier attribution solve?

  • What does event-level admission actually filter out? Can you share concrete examples or stats from the logs showing harmful/redundant actions that were stripped from otherwise successful trajectories?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.