acceptodds
Under review as a conference paper at ICLR 2027

How should experience act? Adaptive experience binding for agent self-evolution

Abstract

Large language model (LLM) agents are increasingly used to handle complex real-world tasks. To reduce the effort of manually designing and optimizing agents, recent work explored , enabling agents to improve themselves toward target tasks automatically. Existing approaches either iteratively search for optimal agent configurations or accumulate and reuse experience to guide subsequent behavior. While the former often requires repeated rollout-and-evaluation cycles, the latter provides a more direct mechanism for preserving and exploiting execution experience. However, existing methods focus on what experiences to learn and retrieve, but leave a critical question largely under-explored: Our measurement studies show that different ways of applying the same experience can substantially influence both agent behavior and end-to-end task performance. To address this gap, we propose , a training-free framework that explicitly regulates when and how learned experience influences agent behavior. AEB represents experience as structured and revisable Binding Experiences, learns and updates them from grounded execution evidence, and dynamically determines the appropriate form and strength of their influence according to the current execution state. Extensive experiments on complex agentic benchmarks demonstrate consistent improvements over representative baselines, with additional analysis experiments further supporting the effectiveness of the proposed design.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Large Language Model Agents, Agent Self-Evolution, Agent Memory, Experience Binding

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_cjiTOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Adaptive Experience Binding (AEB), a training-free framework that explicitly controls when and how retrieved experience influences agent execution. AEB represents experience using structured applicability, satisfaction, reliability, and intervention specifications; updates these specifications from execution evidence; and dynamically selects advice, enforcement, downgrade, or release. Experiments on WebArena and SWE-Bench-Verified show improvements across two backbone models, accompanied by reductions in execution cost.

Critical strengths
  1. The paper identifies a meaningful and actionable distinction between retrieving experience and controlling its influence. The controlled measurement study holds experience fixed and demonstrates complementary failures of insufficient guidance and excessive enforcement. This provides useful insight for designing memory-based agents beyond improving memory extraction and retrieval.

  2. The empirical results support the practical value of the complete framework. Under shared backbones, execution scaffolds, and task budgets, AEB improves performance across two different task domains. On SWE-Bench-Verified with DeepSeek, it improves the resolve rate from ReasoningBank’s 61.2% to 83.8%, while reducing average steps from 71.9 to 31.7. The GPT-4o results also improve, and the appendix reports token and time costs including strategy-side overhead

Critical weaknesses
  1. The boundary between learned binding specifications and predefined executable machinery remains insufficiently specified. Section 4.2 describes experience content and binding specifications as learned and revisable. However, Appendices A.4–A.5 indicate that candidates are constructed by the system, grounded claims use predefined claim types, semantic review accepts or rejects candidates, and runtime assessment is deterministic. These choices are reasonable, but the paper does not sufficiently explain how an evidence capsule becomes an executable condition, which features and condition compositions are supported, or which aspects can genuinely expand through experience. Table 8 provides a conceptual example rather than a complete executable record. This limits reproducibility and the precision of the learning claim.

  2. The main ablation establishes the total benefit of the closed-loop system more clearly than the contribution of its individual mechanisms. In Table 5, each variant independently evolves its memory, so the difference between 83.8% and 72.6% combines immediate binding effects with subsequent changes in learned experience. The frozen-experience study in Section 3.1 is valuable, but it uses only 30 Shopping evaluation tasks and a different experience/control setup. A frozen AEB-store comparison would strengthen the explanation of how much improvement comes from adaptive runtime control versus its interaction with experience updates.

Questions for authors
  1. Could you provide a complete example tracing execution evidence through candidate construction to machine-readable applicability, satisfaction, and enforcement conditions, together with the supported condition language?

  2. With the same frozen AEB experience store, how do advice-only, simple rule-based dynamic binding, and the full Binder compare?

Reviewer_YyEkOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper addresses a central question in agent self-evolution: once an agent has learned from experience, how should that experience shape its behavior at runtime? It introduces Adaptive Experience Binding (AEB), an training-free framework that represents experience as structured, revisable Binding Experiences, continually learns and updates them from grounded execution evidence, and dynamically selects among NOOP, ADVICE, ENFORCE, DOWNGRADE, and RELEASE based on current state, proposed action, and recent history. By mitigating both under-binding and over-binding, AEB improves success rates and execution efficiency on complex agentic tasks.

Critical strengths
  1. Unlike prior work that focuses on improving retrieval accuracy or on how experiences are learned and constructed, this paper centers on a different quesiton: when, in what form, and to what extend should retrieved experience influence agent execution?
  2. Experience binding is still a relatively new concept, and I would agree that effectively exploiting retrieved experience is even more important than improving retrieval accuracy. In my previous work, I often found that even when the retrieved experience was correct, the agent did not necessarily follow it or behave correctly thereafter. This makes further research into experience binding both necessary and worthwhile.
Critical weaknesses
  1. Although AEB achives state-of-the-art results on several metrics on WebArna benchmark, its perforance gains over competing methods are relatively modest.
  2. the experiments are conducted only with GPT-4o and DeepSeek-V4-Flash, without evaluation on newer or more capable models.
  3. The method is relatively complex, and it remains unclear how readily its design can be adopted or extended in future work.
Questions for authors

Please check the section of critical weaknesses.

Reviewer_iR3mOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

AEB is a training-free framework for continual agent improvement through structured experience. It augments memories with applicability, satisfaction, violation, reliability, and intervention specifications; learns and updates them from execution evidence; and uses deterministic runtime logic to choose advisory, enforcing, or relaxed influence. A controlled preliminary study holds memories fixed, while the main evaluation compares complete self-evolving systems on WebArena and SWE-bench Verified with two execution backbones. The reported results show substantial success and efficiency gains, particularly on SWE-bench Verified.

Critical strengths
  1. The problem is meaningful and clearly illustrated. The underuse/overuse distinction captures a real weakness of prompt-only memory. Separating a lesson's content from the conditions and strength of its application is a useful system-design perspective.
  2. The main evaluation is substantial. The experiments cover 684 WebArena tasks and all 500 SWE-bench Verified tasks, with two model settings and shared execution scaffolds. The reported SWE-bench improvement is large enough to warrant serious attention rather than dismissal as an incremental prompting change.
  3. The protocol separates learning signals from benchmark outcomes. Sections 4.3 and A.8 state that learning uses self-judgment from trajectories and never the benchmark evaluator's ground truth. Memory changes are committed only after task completion. These are appropriate boundaries for a continual-learning evaluation.
  4. Several diagnostics go beyond an aggregate leaderboard. Rescue, deterioration, and preservation expose both benefits and disruption. Table 5 includes an advisory-only variant retaining the memory framework, and Appendix A.7 measures aggregate token and wall-clock costs. Runtime determinism also avoids an extra controller-model inference at every step.
Critical weaknesses

W1. The main ablation does not isolate the runtime mechanism claimed to explain the gains. Section 5.3 explicitly states that each binding variant independently runs the full self-evolution process. Its choices therefore change future trajectories, evidence, and stored memories. The 83.8% versus 72.6% full-versus-advisory result measures a system-level intervention, not just the effect of binding strength on the same retrieved experiences. The preliminary fixed-memory WebArena study is useful, but it contains only 30 shopping tasks and uses a simpler rule-based controller. A focused frozen-store comparison of full AEB, advisory-only binding, and the preliminary dynamic rule would directly establish what the learned binding specifications add. Separately, static delivery of the same structured memories would distinguish construction/retrieval gains from runtime intervention.

W2. The executable path from evidence to generalizable binding rules is underspecified. Appendices A.3-A.5 clarify that runtime predicates and binding selection are deterministic, candidate templates are fixed, the first LLM judgment only elaborates a fixed claim type, and the second accepts or rejects a fully constructed candidate. The manuscript also states that the templates are not a predefined content library. Table 8 provides an illustrative structured experience, and Appendix A.4 specifies matching thresholds and update categories; these are useful implementation details. What remains unclear is how new factual evidence becomes executable applicability/satisfaction/violation logic: what is the predicate language, which constructors and claim types are available, and which parts are truly learned versus supplied by the framework? An end-to-end trace from raw event to candidate, accepted rule, predicate evaluation, and later revision is essential to reproduce and assess the claimed self-evolution. Deferring code release is not itself grounds for rejection, but prose descriptions currently leave this central mechanism unresolved.

W3. A single fixed task stream does not establish robust continual improvement. AEB and the baselines use the same order, so the comparison is meaningful for that stream. Nevertheless, memory activation thresholds, repeated repositories, and recurring task templates can make results highly order-dependent. I did not find alternative stream orders or independent-run uncertainty in the supplied manuscript, including Appendices A.6-A.8. The modest GPT-4o WebArena improvement and the large DeepSeek SWE-bench improvement would benefit from a limited second ordering or a frozen-memory transfer slice. This is not an allegation of benchmark-label leakage; the stated learning/evaluation separation should be credited. It is a question about the generality of the sequential-learning claim and comparability with ordinary one-shot SWE-bench results.

W4. Evidence for adaptive specification learning is weaker than evidence for the complete engineered system. Fixed ADVISE already reaches 72.6%, well above ReasoningBank's 61.2%, while ENFORCE collapses to 19.2%. This supports the value of the surrounding memory pipeline and the danger of rigid blocking, but does not show that revisable learned conditions outperform a simple deterministic escalation/backoff policy. Report how many active experiences are genuinely new, how often specifications change, the frequency and outcome of each binding mode, and whether the controller's advantage persists with the simpler policy. These diagnostics would connect the large scores to the proposed learning mechanism rather than only to a general guardrail architecture.

Questions for authors
  1. Can you compare full binding, advisory-only delivery, and the simple dynamic rule using the same frozen experience store and a prespecified evaluation slice?

  2. What executable predicate/condition language and candidate constructors are available? Please provide one complete learned-example trace, including revision after contradictory evidence.

  3. How stable are the key gains under a second task ordering? Can you separate sequential adaptation from generalization to a new task stream?

  4. What are the memory count, activation timing, specification-revision count, and intervention-mode frequencies? How often does an enforcement or downgrade rescue progress versus disrupt it?

  5. Appendix A.7 includes total strategy costs, which is helpful. Please also specify how post-task learning costs relate to the common $3 within-task execution budget, and provide numeric execution/strategy breakdowns rather than only bars.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.