acceptodds
Under review as a conference paper at ICLR 2027

ErRsi: Low-Cost Agent Self-Evolution via Elastic Error Localization

Abstract

Agent recursive self-improvement (RSI) attracts growing interest as a way to improve agents through continued interaction. A crucial component of this process is accumulating and reusing experience to guide subsequent decisions. However, continually adding memories and instructions can increase recurring context costs and introduce irrelevant guidance across task environments. In this work, we propose ErRsi, a two-stage RSI framework with fixed model weights. In the first stage, to extract actionable evidence within context limits, we propose elastic error localization and organize diagnoses and update feedback into error lineages. In the second stage, to translate this evidence into behavioral improvement, we introduce targeted revision with conditional guidance and paired execution for update selection. On Who&When, localization achieves 30.43% exact-step accuracy at 9B and 36.96% at 27B, exceeding representative methods in the matched 9B evaluation. Diagnostic-input comparisons show that localized evidence supports effective revisions. On WebShop, complete-protocol token consumption falls by 44.67–69.77% relative to adapted Agentic Context Engineering (ACE). Frozen WebShop-to-ALFWorld transfer remains within one task of initial success across three model scales, with 75.39–78.64% fewer target-execution tokens than ACE. Together, these results demonstrate accurate error localization, efficient RSI, and low interference across benchmarks.

Primary area:
foundation or frontier models, including LLMs
Keywords:
failure diagnosis, agent self-improvement, error lineages, non-parametric adaptation, executable harnesses

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_8ccrOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes ErRsi, a framework for low-cost agent self-improvement without updating model weights. It combines context-aware error localization, error history organization, and targeted behavioral revision with conditional guidance and paired evaluation. Experiments on Who&When, WebShop, and ALFWorld demonstrate improvements in error localization, task success, and token efficiency.

Critical strengths
  1. The paper addresses a practical and relevant problem: improving agents through execution feedback while controlling the cost and potential interference of accumulated experience.
  2. The framework provides an end-to-end pipeline connecting error diagnosis with behavioral revision, and the use of conditional executable guidance is a practically useful design.
  3. The experiments cover multiple benchmarks and model scales, with encouraging results in token efficiency and task performance.
Critical weaknesses
  1. Limited methodological novelty.

The proposed framework appears to be largely a combination of existing techniques and engineering designs, rather than introducing fundamentally new methods. For example, Elastic Error Localization mainly switches between full-trajectory inspection and active investigation based on context limits, while Dual-View Attribution combines two LLM-based reasoning perspectives with a simple arbitration mechanism. While the overall integration may be practically useful, the paper does not clearly establish what constitutes the key methodological innovation beyond these existing components.

  1. Potential overclaiming of "Elastic Error Localization" and "Recursive Self-Improvement."

The terminology and framing appear stronger than what the proposed methods and experiments substantiate. The "elastic" mechanism is essentially a threshold-based switch between two evidence acquisition strategies, rather than a substantially new localization algorithm. More importantly, the claimed recursive self-improvement is evaluated with only two candidate revision rounds in the main WebShop experiments. This limited setting does not sufficiently demonstrate sustained self-improvement, long-term experience accumulation, or scalability over multiple iterations.

Questions for authors

N/A

Reviewer_3W2eOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces an agent self-improvement framework with fixed model weights. It first localizes errors based on trajectory length and context budget. It then uses error lineages to organize failure mechanisms and feedback from past changes. This information guides revisions to the agent’s external behavior. The learned guidance is applied under specified conditions, and updates are selected through paired-task evaluation. The paper evaluates error localization on Who&When and tests behavioral improvement and execution costs on two benchmarks. It also examines capability retention by freezing the system and deploying it across benchmarks.

Critical strengths

What I find most interesting is the separation between remembering a lesson and applying it when needed. The agent may still ignore a lesson at a key decision point, even when it is in the context. ERRSI turns some lessons into checks that provide corrections when their conditions are met. This lets experience directly influence the next action. This makes me think the method addresses a real problem in how agents use experience. I also appreciate the distinction between the information needed for learning and for execution. Diagnosis and revision can use error histories, local evidence, and feedback from past changes. Only relevant guidance is provided during each execution. The WebShop cost comparison also includes learning calls and evaluations of candidates that were not adopted. So the reported low costs cover the actual improvement process, not just the final agent’s execution.

Critical weaknesses

My main uncertainty is whether ERRSI's learned conditions can tell when past experience still applies in similar new situations. The paper treats “when experience should influence decisions” as a core question. I think this part also deserves closer testing, and the current experiments have not fully resolved my concerns. The comparison with fixed knowledge in the main paper convinces me that executable checks can help decisions on WebShop. The WebShop to ALFWorld results also show that overall performance is largely preserved when these lessons are carried into another environment. But I am more interested in a case between these two settings: the new task looks similar to the original, but past experience only applies in some situations. WebShop and ALFWorld differ in their tasks and action space. Performance close to the initial agent could also mean that the source rules rarely encounter relevant situations in the target environment. I think this possible explanation needs to be checked. Whether this is the reason is not clear from the current results. Testing whether experience remains useful across similar tasks would more directly reveal what these conditions have learned. I would like to see a small controlled experiment using rules the system actually learned. Freeze both the rules and the corrective guidance. Keep similar action interfaces and select tasks or states where the old lessons do and do not apply. Check whether the system intervenes when needed and stays quiet otherwise. Compare this with always providing the same guidance and with disabling that guidance. This would directly affect how we judge condition learning as a core part of the method.

Questions for authors

Could you add the controlled experiment described above? Freeze the learned rules and test them on similar tasks where past experience applies only in some situations. Compare the following three settings: guidance triggered by the learned conditions, the same guidance always provided, and no guidance.

Reviewer_rXXoOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents ErRsi, a fixed-weight agent self-evolution framework with two stages. The first stage uses elastic error localization, switching between dense trajectory analysis and active evidence acquisition according to context constraints, and combines process- and decision-oriented attribution proposals to identify failure origins. Diagnoses are organized into error lineages that associate recurring failure mechanisms and occurrence conditions with behavior versions and update outcomes. The second stage uses these records to revise external agent behavior, including instructions, task procedures, and conditionally triggered execution checks, and selects candidate revisions through paired evaluation against the current behavior. The paper evaluates error localization on Who&When, self-evolution and token consumption on WebShop and ALFWorld, conditional execution of learned guidance, and frozen WebShop-to-ALFWorld transfer across multiple model scales.

Critical strengths
  1. End-to-end pipeline from attribution to deployable revision. Rather than stopping at trajectory diagnosis, ErRsi converts localized evidence into bounded, conditionally triggered guidance and retains revisions only through paired evaluation—a concrete fixed-weight mechanism for improving behavior while controlling when learned experience affects future decisions.

  2. Evaluation that weighs both performance and recurring cost. Beyond asking whether behavior improves, the paper reports complete-protocol token accounting (WebShop), held-out ALFWorld, diagnostic-input controls, and frozen cross-benchmark transfer—examining whether accumulated guidance inflates execution cost or interferes with capabilities in a different environment.

Critical weaknesses

W1. The proposed dual-view attribution is largely prompt ensembling rather than independent reasoning. The process and decision views appear to be two prompt-conditioned attribution proposals generated from the same acquired evidence, rather than genuinely independent perspectives. Since the decision view must also reason about failure propagation to assess whether a local replacement interrupts the failure chain, its reasoning substantially overlaps with the process view. The paper should clarify how candidates are generated, whether the two proposals use the same model and prompting context, and how often they agree or disagree.

W2. The incremental value of error-lineage organization is not empirically isolated. Error lineages, including error-family grouping, condition-specific branches, and links between diagnoses and update responses, are presented as a core component of ErRsi. However, the paper does not compare this structure against simpler alternatives such as a flat diagnosis history, latest-only feedback, or no update history. The existing attribution and diagnostic-input ablations do not isolate lineage organization, and the WebShop evaluation contains only two candidate-update rounds. Consequently, the results validate the overall diagnosis-guided revision pipeline, but do not establish whether error lineages themselves provide incremental benefits.

W3. The headline token-efficiency gains are established only against a single full-playbook ACE adaptation. The reported 44.67–69.77% token savings are obtained against a single ACE adaptation with a structurally different knowledge-deployment mechanism. ErRsi provides learned corrective guidance only when executable checks fire, whereas the ACE actor receives an up-to-8,192-token full playbook at every decision, creating a built-in source of recurring context cost for ACE. While this is a valid system-level comparison, it does not isolate how much of the saving comes from selective deployment rather than error localization or lineage construction. No matched compact-memory or retrieval-based baseline is evaluated under the complete RSI protocol, and the compressed-ACE control in Appendix F.1 covers only 32 tasks, compresses a single frozen playbook, and does not retrain ACE under the reduced capacity. Therefore, the results establish that ErRsi is cheaper than this ACE configuration, but the broader efficiency advantage over alternative RSI designs remains under-supported.

Questions for authors
  1. How does ErRsi behave under long-horizon experience accumulation? The paper motivates ErRsi through continuing workloads in which experience and recurring context costs accumulate over many interactions, but the complete WebShop protocol contains only two candidate-update rounds and explicitly does not measure a long-run accumulation curve. Could the authors provide a longer rolling evaluation, or additional evidence showing how lineage size, executable rules, token cost, and behavioral interference evolve over many successive revisions?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.