acceptodds
Under review as a conference paper at ICLR 2027

More Memory, Worse Agents: Error Reproduction and Anti-Persistence in LLM Agents

Abstract

LLM agents can improve across tasks without updating model parameters by storing experience from their own executions in external memory and reusing it to guide later decisions. A central design question is whether to retain only observed trajectories and outcomes or also the rules, summaries, and skills inferred from them. Retaining these abstractions allows agents to reuse strategies without repeatedly deriving them from raw trajectories. However, errors in stored abstractions can propagate to subsequent decisions through retrieval and reuse. They can also shape execution trajectories, which then become the basis for new abstractions that reproduce the original errors. We term this feedback mechanism error reproduction: errors are regenerated through a cycle of abstraction, execution, and re-abstraction, extending their persistence beyond direct reuse of the original faulty knowledge. We propose Anti-Persistence, a memory design that accumulates trajectories and outcomes while reconstructing task-specific guidance on demand. An adaptive selector uses prior outcomes to decide whether and how to synthesize guidance, which is discarded after execution. This prevents its direct reuse as a stored object, although indirect feedback through retained trajectories remains possible. Across different benchmarks and backbones, Anti-Persistence achieves higher overall held-out success and the lowest average LLM-call cost among the evaluated memory designs. These results support separating experience accumulation from abstraction persistence in memory-based self-improvement.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Large Language Models, AI Agents, Agent Memory, Agentic Systems

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_NWKvOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper argues that agents which store abstractions of past experience (rules, strategies, skills) tend to reproduce their own errors: a bad abstraction shapes future trajectories, which are then abstracted again. As an alternative it proposes ANTI-PERSISTENCE. Memory keeps only raw trajectories and outcomes, and guidance for each new task is generated on the fly and then thrown away. A bandit-style selector picks one of four guidance modes per task. Experiments on ALFWorld, WebShop and ScienceWorld with three open models compare the method against ReAct, Reflexion, ExpeL, AutoRefine and trajectory, abstraction and joint memory variants. It has the best average success rate, and discarding guidance beats keeping it in a direct comparison. Ablations cover the mode selector and its hyperparameters.

Critical strengths
  • The paper tests a clear design question under a controlled protocol: keep inferred guidance, or rebuild it from stored trajectories for each task. All methods share fixed train/held-out partitions and memory is frozen at evaluation. Under this setup the method beats trajectory and abstraction memory in all 9 settings and joint memory in 8 (Table 1), and abstraction and joint memory trail trajectory memory in 7 of 9.

  • The mode selector is ablated sensibly. In Fig. 6 the bandit beats fixed modes and random selection on Qwen, and the best fixed mode differs by benchmark. Fig. 8 shows the zero-cost none mode is used in every setting.

Critical weaknesses
  • The error-reproduction claim in the title and Contribution 3 is not tested in the main text. Contribution 3 promises "associations between earlier and subsequently generated abstractions", but Section 4 defers this to Sections E.3 and F. Nothing in Section 4 tracks whether an error in one abstraction reappears in a later one.

  • The baselines seem to be under-specified. Section 4.1 only names the four agent baselines. ExpeL on Llama/ALFWorld scores 7.2 against 29.3 for ReAct, which suggests a setup problem. The main text does not say how Reflexion, the strongest baseline at 52.9 on average, works on held-out tasks when memory is frozen during evaluation.

Questions for authors
  • How were gamma = 0.998 and lambda = 0.05 chosen, and on which split were Figs. 5 and 7 computed? If on the held-out sets, please redo Table 1 with values picked on a split of the training tasks and add seed-level significance tests. A gain that holds up would raise my soundness score.
  • What evidence in Section 4 shows error reproduction specifically, rather than a general benefit of on-demand synthesis? For example, seed a known faulty rule, delete it, and count how often the error reappears in later guidance under persistent memory vs. anti-persistence.
  • In Table 2, how is persisted guidance stored, indexed and retrieved, and how does Persist differ from Abs. Mem.? Why does Persist on Llama/ALFWorld (26.7) fall below ReAct (29.3)?
  • For Llama/ScienceWorld, what fraction of episodes end early under each memory design, and why (parse errors, context length, invalid actions)? If most of the joint-memory drop comes from such failures, the result is not about error reproduction.
  • Does Reflexion get multiple trials per held-out task, and does its reflection memory carry across tasks during evaluation? Why does ExpeL on Llama/ALFWorld (7.2) score far below ReAct (29.3)?
3 comments
AuthorsOpenReview ↗

To Reviewer NWKv (1/3)

Q1: The error-reproduction claim in the title and Contribution 3 is not tested in the main text. Contribution 3 promises "associations between earlier and subsequently generated abstractions", but Section 4 defers this to Sections E.3 and F. Nothing in Section 4 tracks whether an error in one abstraction reappears in a later one.

A1: We will add the controlled lineage experiment to the main text and update Contribution 3 accordingly. Building on the Useful/Toxic/None paired design in Appendix F.1, the experiment checks whether specific errors recur in later executions and appear in newly generated rules.

Using DeepSeek-V4-Flash on AppWorld, we compared 12 incorrect rules (Toxic), their matched corrections (Useful), and no injected rules (None). Each pair differed in a single fact. All conditions used the same 30 tasks, three random seeds, and three rounds of skill evolution. Initial rules remained available during execution. The skill evolver received execution traces and existing skill names, with the rule text withheld.

Dedicated detectors checked the generated code for each target error, while a judge applied the same criteria across conditions to check whether newly generated rules restated that error.

ConditionExecution error recurrence (round 0 → round 3)New rules restating a target error
None: no injected rules42% → 24%1.4%
Useful: matched corrected rules31% → 27%2.8%
Toxic: incorrect rules70% → 57%4.6%

After three rounds, execution error recurrence was 30 % higher in Toxic than in Useful. Some newly generated rules also restated the same target errors. The main text will present both execution-level and content-level checks, with the text-recurrence rates reported descriptively.

Q2: The baselines seem to be under-specified. Section 4.1 only names the four agent baselines. ExpeL on Llama/ALFWorld scores 7.2 against 29.3 for ReAct, which suggests a setup problem. The main text does not say how Reflexion, the strongest baseline at 52.9 on average, works on held-out tasks when memory is frozen during evaluation.

A2: Our comparisons use shared base execution settings while preserving each baseline’s algorithmic procedures. All methods use the same backbone configuration, training and held-out partitions, training tasks per iteration, base agent prompt, and per-step output budget. Each episode permits at most 50 environment steps. These settings are specified in §4.1 and Appendix D.

Reflexion’s task-local reflection and frozen memory operate at different scopes. Each held-out task allows up to three attempts: the initial attempt plus two retries. Reflections generated after failure are used only for subsequent attempts on that task and cleared afterward. Cross-task memory accumulated during training remains frozen throughout evaluation. New trajectories, outcomes, and reflections do not carry across held-out tasks. Task-local reflection is therefore compatible with frozen cross-task memory.

ExpeL’s 7.2% is the result obtained under these shared settings. Beyond base execution, ExpeL extracts rules from experience and uses those rules alongside retrieved examples on later tasks, making its benefits dependent on the backbone’s ability to process experience. As Table 1 shows, ExpeL underperforms ReAct on Llama/ALFWorld but outperforms it with Qwen3-Coder-Next and Gemma-4-31B-it. This demonstrates backbone-dependent performance. A low score in one setting alone does not establish a configuration error.

AuthorsOpenReview ↗

To Reviewer NWKv (2/3)

Q3: How were gamma = 0.998 and lambda = 0.05 chosen, and on which split were Figs. 5 and 7 computed? If on the held-out sets, please redo Table 1 with values picked on a split of the training tasks and add seed-level significance tests. A gain that holds up would raise my soundness score.

A3: We selected γ = 0.998 and λ = 0.05 through preliminary tests on a small-scale dataset, then fixed them before held-out evaluation and used the same values across all Table 1 settings. The former gradually discounts older feedback; the latter mildly penalizes extra synthesis calls. Figures 5 and 7 are subsequent held-out sensitivity analyses and did not determine the defaults. We will clarify the pilot dataset and selection procedure in the revised manuscript.

We also ran additional experiments with DeepSeek-V4-Flash, fixing the code defaults γ = 0.995 and λ = 0.05 without further tuning. Both used three seeds and a single final test evaluation with memory frozen. AppWorld used 20 training and 50 test tasks; WebWalkerQA used 300 distinct training tasks and 150 test tasks from unseen websites.

Memory configurationAppWorld success (%)WebWalkerQA success (%)
Trajectory Memory88.040.2
Abstraction Memory93.345.1
Joint Memory96.045.3
Anti-persistence92.748.2

We added two-sided exact seed-level paired sign tests, comparing success-rate differences within each seed and excluding ties:

BaselineAppWorld wins/losses (p)WebWalkerQA wins/losses (p)
Trajectory Memory2/0 (0.50)3/0 (0.25)
Abstraction Memory1/2 (1.00)2/0 (0.50)
Joint Memory1/2 (1.00)2/0 (0.50)

With three seeds, the smallest attainable two-sided p-value is 0.25, so none reached significance. On WebWalkerQA, Anti-persistence matched or exceeded each memory baseline in every seed. Full results and statistical details will be included in the revised manuscript.

Q4: What evidence in Section 4 shows error reproduction specifically, rather than a general benefit of on-demand synthesis? For example, seed a known faulty rule, delete it, and count how often the error reappears in later guidance under persistent memory vs. anti-persistence.

A4: To examine specific error content, we extended the Useful/Toxic/None paired design in Appendix F.1 into a controlled lineage experiment, which we will report in Section 4. It checks whether the same target errors recur in later executions and appear in newly generated guidance.

Using DeepSeek-V4-Flash on AppWorld, we compared 12 incorrect rules (Toxic), their matched corrections (Useful), and no injected rules (None). Each pair differed in a single fact. All conditions used the same 30 tasks, three random seeds, and three rounds of skill evolution. Initial rules remained available during execution. The skill evolver received execution traces and existing skill names, with the rule text withheld.

Dedicated detectors checked the generated code for each target error, while a judge applied the same criteria across conditions to check whether newly generated rules restated that error.

ConditionExecution error recurrence (round 0 → round 3)New rules restating a target error
None: no injected rules42% → 24%1.4%
Useful: matched corrected rules31% → 27%2.8%
Toxic: incorrect rules70% → 57%4.6%

After three rounds, execution error recurrence was 30 percentage points higher in Toxic than in Useful. Some newly generated rules also restated the same target errors. These checks examine both execution behavior and specific content in new abstractions, providing evidence alongside the success-rate comparisons. The text-recurrence rates are reported descriptively.

AuthorsOpenReview ↗

To Reviewer NWKv (3/3)

Q5: In Table 2, how is persisted guidance stored, indexed and retrieved, and how does Persist differ from Abs. Mem.? Why does Persist on Llama/ALFWorld (26.7) fall below ReAct (29.3)?

A5: Table 2 is a direct ablation under a shared configuration. Discard and Persist use identical trajectory stores, bandits, cost weights, synthesis prompts, and execution settings. They differ in whether generated guidance is retained and reused.

During training, Persist appends fresh guidance to JSONL with task information, mode, outcome, iteration, and task embedding. Guidance from successful and failed episodes is retained with outcome labels. From the second iteration onward, whenever synthesis is invoked, Persist retrieves the top-2 items of the same task type from earlier iterations by cosine similarity and prepends them to the fresh guidance. Only newly generated text is stored, and the store remains frozen during evaluation.

PersistAbs. Mem.
Stored contentTask-specific guidanceGeneral skills extracted by an evolver
RetrievalTop-2 when synthesis is invokedTop-5 every episode
Fresh task-time synthesisYesNo
Trajectory store retainedYesNo

On Llama/ALFWorld, Persist and ReAct share the backbone and base evaluation settings, with a mean difference of 2.6 percentage points (26.7% vs. 29.3%). Guidance from earlier tasks, including failed episodes, may interfere with current decisions. The direct ablation shows a larger difference: 54.7% for Discard versus 26.7% for Persist, a mean advantage of 28.0 percentage points, supporting discarding synthesized guidance after use in this setting.

Q6: For Llama/ScienceWorld, what fraction of episodes end early under each memory design, and why (parse errors, context length, invalid actions)? If most of the joint-memory drop comes from such failures, the result is not about error reproduction.

A6: A6. Assuming at least one LLM call per environment step and a maximum of 50 steps, we calculated the following bounds on budget exhaustion from Tables 1 and 6.

Memory configurationSuccess rateBudget exhaustion
Trajectory Memory20.0%≤7.50%
Abstraction Memory18.9%≤2.66%
Joint Memory7.2%≤2.30%
Anti-persistence30.6%≤7.12%

Under these assumptions, Joint Memory’s budget-exhaustion rate is at most 2.30%, so most failures occur before the step budget is exhausted, consistent with the possibility of early failure discussed in §4.3. The aggregate metrics cannot distinguish incorrect completion from termination caused by parsing errors, context limits, or invalid actions. These results alone therefore cannot establish whether error reproduction caused the performance decline.

Q7: Does Reflexion get multiple trials per held-out task, and does its reflection memory carry across tasks during evaluation? Why does ExpeL on Llama/ALFWorld (7.2) score far below ReAct (29.3)?

A7: All methods share the model configuration, task partitions, base agent prompt, and per-step output budget (§4.1, Appendix D), while retaining their respective memory and reflection procedures.

Reflexion. Each held-out task permits up to three attempts. New reflections are used only for retries on that task and cleared afterward. They do not carry across tasks, and persistent memory accumulated during training remains frozen during evaluation.

ExpeL. Its 7.2% on Llama/ALFWorld and ReAct’s 29.3% were obtained under the same base configuration. In Table 1, ExpeL outperforms ReAct on ALFWorld with the other two backbones, showing backbone-dependent benefits. The low score alone does not establish a configuration error.

Reviewer_WVL4OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper argues that persisting abstractions such as rules and skills can propagate errors through subsequent actions and the trajectories used to generate later abstractions. It proposes Anti-Persistence: preserve trajectories and outcomes, but generate guidance only for the current task and discard it afterward. A task-conditioned selector chooses among no synthesis, imitation, strategic guidance, and corrective guidance using historical outcomes, decay, exploration, and a cost penalty.

Critical strengths

Pros

  1. The paper separates the value of abstraction from the value of retaining it. Temporary guidance can adapt to current tasks. Stored experience remains available for later reinterpretation.

  2. The evaluation covers three models, three environments, three seeds, and ten update rounds. Freezing memory and controller states prevents test outcomes from affecting later updates.

  3. The selector balances different uses of experience. It chooses demonstrations, strategic advice, corrective guidance, or no synthesis for each task. Comparisons with fixed and random modes, together with sensitivity analyses, support selective guidance generation.

Critical weaknesses

Cons

  1. Evidence for error reproduction is weaker than evidence for performance gains. Error labels may reflect task difficulty or retrieval bias, and the supporting aggregates lack verified event-level provenance.

  2. The comparisons do not establish a general advantage over reliable skill reuse. AutoRefine changes several components, while retention experiments have limited coverage and uncertainty reporting.

Questions for authors

Questions

  1. Can a controlled lineage experiment test error reproduction?

  2. When would verified persistence outperform unconditional discarding?

2 comments
AuthorsOpenReview ↗

To Reviewer WVL4 (1/2)

Q1: Evidence for error reproduction is weaker than evidence for performance gains. Error labels may reflect task difficulty or retrieval bias, and the supporting aggregates lack verified event-level provenance.

A1: To directly address task difficulty and retrieval bias, we added a controlled AppWorld experiment with matched tasks, controlled rule exposure, and error-specific checks.

Using DeepSeek-V4-Flash, we compared 12 incorrect rules (Toxic), their matched corrections (Useful), and no injected rules (None). Each pair differed in a single fact. All conditions used the same 30 tasks, three random seeds, and three rounds of skill evolution. Every episode received its condition’s full skill library, removing selective retrieval as a confound.

We checked each episode’s generated code for the specific target errors using dedicated detectors. Separately, a judge examined whether newly generated rules restated those errors.

ConditionExecution error recurrence (round 0 → round 3)New rules restating a target error
None: no injected rules42% → 24%1.4%
Useful: matched corrected rules31% → 27%2.8%
Toxic: incorrect rules70% → 57%4.6%

After three rounds, Toxic showed 30 % higher execution error recurrence than Useful on the same tasks. The targeted errors also appeared in some newly generated rules. These controlled, content-specific checks strengthen the evidence beyond the descriptive provenance associations in §E.3. The text-recurrence rates are reported descriptively.

Q2: The comparisons do not establish a general advantage over reliable skill reuse. AutoRefine changes several components, while retention experiments have limited coverage and uncertainty reporting.

A2: Table 2 tests the effect of retaining guidance more directly. Both variants generate and use guidance for the current task; they differ in whether that guidance is discarded afterward or stored for later retrieval. Across four settings on ALFWorld and WebShop, discarding guidance improved mean success by 3.3–28.0%. We report the mean and standard deviation across three seeds.

We also tested whether screening skills before retaining them helps. Appendix F.2 reports this comparison on WebShop. We extended it to WebWalkerQA with a different backbone.

Held-out benchmarkBackboneAnti-persistenceWith additional judge-screened persistent skills
WebShop (original paper)Qwen3-Coder-Next56.0%44.0%
WebWalkerQA (new)DeepSeek-V4-Flash48.2%42.0%

Adding the screened skills reduced mean success in both settings. These controlled comparisons provide evidence for the tested retention policies independently of the AutoRefine comparison, whose component differences are documented in §D.3.

AuthorsOpenReview ↗

To Reviewer WVL4 (2/2)

Q3: Can a controlled lineage experiment test error reproduction?

A3: We ran a controlled lineage experiment on AppWorld using DeepSeek-V4-Flash. We compared 12 incorrect rules (Toxic), their matched corrections (Useful), and no injected rules (None). Each pair differed in a single fact. All conditions used the same 30 tasks, three random seeds, and three rounds of skill evolution. The skill evolver received execution traces and existing skill names, with the rule text withheld.

Dedicated detectors checked the agent’s generated code for each target error. Separately, a judge checked whether newly generated rules restated those errors, using the same reference errors and criteria across all conditions.

ConditionExecution error recurrence (round 0 → round 3)New rules restating a target error
None: no injected rules42% → 24%1.4%
Useful: matched corrected rules31% → 27%2.8%
Toxic: incorrect rules70% → 57%4.6%

After three rounds, execution error recurrence was 30 % higher in Toxic than in Useful. The same target errors also appeared in some newly generated rules, providing content-level evidence alongside the execution results. The text-recurrence rates are reported descriptively.

Q4: When would verified persistence outperform unconditional discarding?

A4: We expect verified persistence to offer an advantage when skills have passed execution tests, their applicability conditions are explicit and checkable, and they are frequently reused in a stable environment. For example, an operation validated for a specific API version can be reused on matching tasks, reducing repeated synthesis and the risk of introducing errors during regeneration. Persistence becomes advantageous when these benefits exceed verification and maintenance costs.

Appendix H explicitly acknowledges the benefits of reusing reliable knowledge and proposes external verification for selective persistence. Appendix F.2 and Figure 11(b) evaluate a GPT-5.5-screened retention strategy. Together, these motivate verification that checks both a rule’s correctness and its applicability to the current task. Combining execution validation with applicability checks is the setting in which we expect verified persistence to be most beneficial.

Reviewer_wPsROpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies error reproduction in memory-based self-improving agents, where faulty abstractions influence subsequent trajectories and are regenerated through later memory updates. It proposes Anti-Persistence, which retains trajectories and outcomes but generates task-specific guidance on demand and discards it after execution. A contextual bandit selects between direct trajectory reuse and three synthesis modes.

Critical strengths
  1. The distinction between accumulating experience and persisting inferred abstractions raises a relevant question for agent memory design.
  2. Experiments cover three benchmarks and three backbones, with improvements across most settings.
  3. The persist-versus-discard ablation provides useful evidence for the proposed design, and the evaluation protocol separates memory updates from held-out evaluation.
Critical weaknesses
  1. Main experiments use ALFWorld, WebShop, and ScienceWorld, with small, repeatedly sampled training pools. A search or browsing benchmark such as WebWalkerQA or GAIA would better test whether the findings generalize to more diverse tasks.
  2. Missing relevant baselines. ReasoningBank, Dynamic Cheatsheet, HyperSkill, and Memp directly address experience abstraction, retrieval, or maintenance. Simplified abstraction and joint-memory baselines cannot adequately represent these approaches.
  3. Unclear distinction from DC-RS. Dynamic Cheatsheet’s DC-RS already retrieves historical examples and synthesizes guidance before execution. Comparing against a variant that does not carry forward the previous cheatsheet would clarify the contribution of anti-persistence beyond existing retrieval-and-synthesis methods.
  4. The parent–descendant relation only indicates that an earlier rule appeared in the context of a trajectory later used to generate a new rule. It does not verify whether the earlier rule influenced execution or whether the new rule reproduced the same erroneous content. Thus, the higher rate of flagged descendants among those with flagged parents establishes an association, but does not confirm the proposed error reproduction mechanism.
Questions for authors
  1. Can you verify whether descendant abstractions reproduce the same specific errors as their parents, rather than merely sharing provenance and being associated with lower success rates?
2 comments
AuthorsOpenReview ↗

To Reviewer wPsR (1/2)

Q1: Main experiments use ALFWorld, WebShop, and ScienceWorld, with small, repeatedly sampled training pools. A search or browsing benchmark such as WebWalkerQA or GAIA would better test whether the findings generalize to more diverse tasks.

A1: Following the reviewer’s suggestion, we added a WebWalkerQA experiment to the revised manuscript (starting at line XXX). We evaluated the four memory configurations from our main experiments with DeepSeek-V4-Flash across three random seeds. To address repeated sampling, we used 300 distinct training tasks, introducing 30 new tasks in each of 10 rounds. The held-out set comprised 150 tasks from websites absent from the training set, allowing us to test whether accumulated experience transfers to unseen websites.

Memory configurationWebWalkerQA held-out success rate (%)
Trajectory Memory40.2 ± 2.2
Abstraction Memory45.1 ± 1.7
Joint Memory45.3 ± 2.0
Anti-persistence (ours)48.2 ± 0.8

Anti-persistence achieved the highest mean success rate, 2.9% above Joint Memory, the strongest baseline in this comparison.

Q2: Missing relevant baselines. ReasoningBank, Dynamic Cheatsheet, HyperSkill, and Memp directly address experience abstraction, retrieval, or maintenance. Simplified abstraction and joint-memory baselines cannot adequately represent these approaches.

A2: Thank you for pointing out these relevant baselines. Abstraction Memory and Joint Memory were designed to compare memory configurations within a shared framework. In the revised manuscript (starting at line XXX), we added direct comparisons with ReasoningBank and Dynamic Cheatsheet on WebWalkerQA. All methods used the same backbone, training task stream, and held-out set, with three random seeds.

MethodWebWalkerQA held-out success rate (%)
Anti-persistence (ours)48.2 ± 0.8
Dynamic Cheatsheet (DC-RS)47.3 ± 2.5
ReasoningBank47.1 ± 3.0
Dynamic Cheatsheet (DC-Cu)44.7 ± 2.4

Anti-persistence achieved a mean success rate 0.9 % higher than DC-RS and 1.1 % higher than ReasoningBank.

Q3: Unclear distinction from DC-RS. Dynamic Cheatsheet’s DC-RS already retrieves historical examples and synthesizes guidance before execution. Comparing against a variant that does not carry forward the previous cheatsheet would clarify the contribution of anti-persistence beyond existing retrieval-and-synthesis methods.

A3: Both DC-RS and Anti-persistence retrieve historical experience and synthesize guidance before execution. They differ in guidance persistence and adaptive synthesis selection. DC-RS combines the previous cheatsheet, the current task, and retrieved examples to produce a new cheatsheet, which it retains. Anti-persistence uses the current task and outcome log to decide whether and how to synthesize guidance, then discards any synthesized guidance after execution (§3.2).

Following the reviewer’s suggestion, we added DC-RS-no-carry, which preserves DC-RS’s retrieval and pre-execution synthesis but generates each cheatsheet solely from the current task and retrieved examples. All methods used DeepSeek-V4-Flash.

MethodAppWorld held-out success (%)WebWalkerQA held-out success (%)AppWorld auxiliary callsWebWalkerQA auxiliary calls
Anti-persistence92.748.20.20 / 0.310.25 / 0.33
DC-RS88.747.31.90 / 1.001.89 / 1.00
DC-RS-no-carry81.346.90.90 / 1.000.90 / 1.00

Auxiliary LLM calls are averaged per episode and reported as training / evaluation. The total call count in §4.2 also includes execution calls.

Removing cheatsheet carryover reduced DC-RS’s mean success by 7.4 % on AppWorld and 0.4 % on WebWalkerQA. Among these three methods, Anti-persistence achieved the highest mean success and used the fewest auxiliary LLM calls on both benchmarks.

AuthorsOpenReview ↗

To Reviewer wPsR (2/2)

Q4: The parent–descendant relation only indicates that an earlier rule appeared in the context of a trajectory later used to generate a new rule. It does not verify whether the earlier rule influenced execution or whether the new rule reproduced the same erroneous content. Thus, the higher rate of flagged descendants among those with flagged parents establishes an association, but does not confirm the proposed error reproduction mechanism.

A4: The provenance analysis in §E.3 records that a parent rule appeared in the execution context and that the resulting trajectory was later used to generate a new rule. To further examine behavioral influence and error-content recurrence, we extended the Useful/Toxic/None paired design in Appendix F.1 to three rounds of skill evolution.

Using DeepSeek-V4-Flash, we compared 12 incorrect rules, their matched corrections, and no injected rules. Each pair differed in a single fact. All three conditions used the same 30 AppWorld tasks and three random seeds. The skill evolver received execution traces and existing skill names, with the rule text withheld.

Dedicated detectors checked the generated code for each target error, while a judge checked whether newly generated rules restated that error.

ConditionExecution error recurrence (round 0 → round 3)New rules restating a target error
None: no injected rules42% → 24%1.4%
Useful: matched corrected rules31% → 27%2.8%
Toxic: incorrect rules70% → 57%4.6%

After three rounds, execution error recurrence was 30 % higher in Toxic than in Useful, and some newly generated rules restated the target errors. The execution results support the influence of initial rule content on later behavior; the text-recurrence rates are reported descriptively. Because the initial rules remained available, execution recurrence also includes their continued influence.

Q5: Can you verify whether descendant abstractions reproduce the same specific errors as their parents, rather than merely sharing provenance and being associated with lower success rates?

A5: We performed content-level checks targeting specific errors in newly generated rules. Using the 12 incorrect rules and their matched corrections as references, a judge assessed whether each new rule restated the same target error. All three conditions used the same criteria.

In the Toxic condition, 4.6% of new rules restated a target error, compared with 2.8% in Useful and 1.4% in None. These checks identified the same specific errors reappearing in new abstractions, providing content-level evidence that complements the execution-error results. These rates are reported descriptively.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.