acceptodds
Under review as a conference paper at ICLR 2027

Don’t Forget What You Just Learned: Self-Directed Repair Makes Agent Skills Persist

Abstract

Intelligent agents should learn from their mistakes so experience compounds into continuous improvement. Yet today’s agents recover from a mistake only temporarily: in-context reflections are easily forgotten across sessions and compete for limited context, while full parameter fine-tuning is expensive and can interfere with existing capabilities. We introduce Self-Direction, a mechanism for agents to learn from mistakes that combines the benefits of temporary retries in-context and policy updates. When an agent fails a task, it reflects on its own trajectory in order to extract a lesson and immediately distills it into a lightweight low-rank adapter. By keeping the original weights frozen, Self-Direction preserves existing capabilities while only training \ 0.02% additional parameters for each adapter. The agent then saves these adapters into a persistent skill library, automatically routing them to solve related problems later. Experiments with open agents based on the Qwen-family across open-ended research (GAIA), tool-use workflows (Claw-Eval), and terminal tasks (Terminal-Bench 2.1) show our method improves overall success rates by 8.9 to 17.0 percentage points and performs competitively with frontier systems. We find Self-Direction avoids repeating mistakes across future encounters of a task, nearly doubling the reliability of an agent’s corrections and enables transfer of learned skills with just a handful of adapters generalizing to dozens of new tasks without additional training. Ultimately, this self-directed repair transforms everyday mistakes into lasting, reusable capabilities.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
reliability, continual learning, agents, persistence, online learning, self-improvement, skills

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_3JstOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Self-Direction, a framework that enables agents to learn from their own failures and retain corrective experience through lightweight LoRA adapters. The experiments examine task performance, repeated-use reliability, and transfer to unseen tasks. I think it is a very good try to compare the memory-style method v.s. parameter-based method for self-reflection.

Critical strengths
  1. This is a very interesting idea that explores an important direction for continual agent improvement. In particular, the results suggest that parametric memory can be a promising alternative to textual memory for retaining and applying procedural experience.
  2. The paper also provides useful experiments beyond aggregate task success, including comparisons with textual memory and pooled adapters, repeated-encounter evaluations, and cross-task transfer.
Critical weaknesses
  1. Failures provide useful signals about what needs improvement, but successful trajectories can also contain reusable procedures, effective tool-use strategies, and robust verification practices. Why not choose the successful trajectories? Or just pairwise learning to train the adaptor?
  2. The paper considers a capped bank and similarity thresholds, which provide useful initial controls. However, these do not fully address how memory should be maintained over a long interaction history: adapters may become redundant, overlap in functionality, or become outdated. It remains unclear when adapters should be consolidated, updated, or removed.
Questions for authors
  1. Is failure-triggered learning primarily an efficiency choice, or is there evidence that failure-derived experience produces more useful adapters than successful experience under comparable budgets?
  2. How does performance change as the adapter bank grows? Can the method identify redundant adapters and retain those with the greatest utility across tasks?
  3. Would routing based on tool-call sequences, error types, or failure-trajectory representations outperform task-description similarity?
2 comments
AuthorsOpenReview ↗

Response to the reviewer 3's review, supported with relevant clarifications and additional experiments and updates to the paper

Responses to the critical weaknesses

We thank the reviewer for recognizing the promise of parametric memory and the value of evaluating persistence and transfer beyond aggregate task success.

1. Why focus on failures instead of successful trajectories or pairwise learning

This is a very interesting idea and we would like to thank the reviewer for bringing it up for discussion. The setting that we aim to explore in the current paper is mistake-conditioned, since we focus on examining and fixing how agents recover from failures. For the purposes of this project, we treat failures as the intervention point because they offer learning opportunities beyond the current capabilities of the model and learning something new that the current model lacks. This holds even for a stronger base model, as Qwen3.8-27B fails fewer tasks, yet Self-Direction still adds 16.4 points on GAIA (App. D). It is also an efficiency choice, since adapters are only trained for the tasks that the base model fails (e.g., 101 of 165 GAIA tasks). In short, the failure trigger directs learning toward observed weaknesses and remains useful as the base model improves. We agree with the reviewer that successful trajectories also offer a learning signal and have acknowledged this in the new discussion section of the updated paper. All changes in the updated paper are highlighted in blue.

2. Maintaining the adapter library over a long interaction history

In the GAIA stream, varying the routing threshold produces libraries of 3, 6 and 12 adapters. Final accuracy remains similar, while new training is avoided for 97.1%, 93.5% and 88.7% of failures, respectively (Section 4.4, App. C.7). Cross-benchmark tests with full libraries also preserve existing capabilities on GAIA and Terminal-Bench for τ ≥ 0.30 (App. C.8). These results show that compact libraries can substantially reduce new training requirements while maintaining similar final accuracy in the tested streams.

Motivated by the reviewer’s comment, we conduct additional analysis on the utility of the adapters. In the 12-adapter library, 4 adapters account for all 16 reuse wins, while the other 8 were retrieved 20 times without a win. Since we log the retrievals and wins of every adapter, these counts directly identify redundant adapters, and a simple utility-based policy can follow naturally, e.g., removing adapters that are retrieved repeatedly without a win or consolidating adapters with overlapping source tasks. We agree that this is important for long interaction histories and have highlighted this in the new Discussion Section and Appendix K.2 of the updated paper.

Additional Clarifications

1. Could tool calls, error types or failure trajectories improve routing

This is a great suggestion. We route on task descriptions because they are simple and available for every task, but we agree that the reviewer's alternatives could offer a strong retrieval too. Successful transfers in the GAIA stream share a procedure rather than a topic (Table 14), and same-level and cross-level retrievals succeed at nearly the same rate (14.7% vs. 14.5%, App. H.1). Since retrieval only happens after a failure, the failed trajectory and its tool-call sequence are already available at routing time at no extra cost. Error types could complement the task embedding. Motivated by the reviewer’s question, we have added this to the new discussion section of our paper.

Thanks for the author's rebuttal. I encourage the author to include those discussions in the final version. I will keep my positive scores.

Reviewer_Y2PDOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Self-Direction aims to make an agent’s corrections persist across sessions. After a verified failure, the agent uses its own trajectory to generate procedural instruction-response examples and trains a small LoRA adapter while keeping the base model frozen. The adapter is stored in a persistent library and retrieved for subsequent failures on similar tasks. If the adapter-guided attempt fails, the system falls back to another base-model attempt.

Experiments on GAIA, Claw-Eval, and Terminal-Bench assess immediate recovery, consistency across repeated encounters, and transfer to unseen tasks. The paper reports improvements over base execution and textual-reflection baselines, alongside experiments with frozen adapter libraries and stronger base models. Its main contribution is the integration of failure-driven self-supervision with separately stored, reusable parametric repairs.

Critical strengths

The strongest evidence concerns persistence and transfer. Table 1 compares repeated encounters with the same two-rollout allowance: success across all three encounters rises from 9.7% to 18.9% over 185 prior failures. The frozen-library GAIA experiment provides complementary evidence of generalization: adapters trained on Level-1 failures raise Level-2 accuracy from 40.7% to 51.2%, compared with 43% for base pass@2. These results support benefits beyond obtaining another chance to solve a task.

The appendix contains substantial supporting analysis, including eight Terminal-Bench runs, pooled and mismatched adapter controls, random versus SVD initialization, routing-threshold sweeps, and measured construction costs. Appendix F also specifies and audits the categorical feedback exposed to reflection, addressing a potential source of answer leakage.

Critical weaknesses

The experiments do not cleanly establish the benefit of storing lessons in weights rather than retrieving them as text. This distinction is central to the contribution, yet Figure 6 shows different initial base success rates across conditions, and the paper does not clearly establish that textual baselines receive identical corrective examples, retrieval decisions, and fallback opportunities. The overall GAIA advantage over Skills.md is only 2.5 percentage points, without a paired uncertainty analysis for that comparison. The existing adapter ablations are useful, but do not resolve this issue. A matched comparison on a common failure subset would determine whether the parametric representation itself provides a reliable advantage. Without it, the central comparative claim remains insufficiently supported.

The evidence for continual improvement through a growing repair library is limited. The repeated-encounter experiment supports persistence on previously seen tasks, and the frozen Level-1-to-Level-2 GAIA experiment supports transfer. However, the three-adapter deployment result combines adapter reuse with base fallback, while the Claw-Eval category-transfer gain amounts to three recovered tasks. These results leave uncertainty about how reliably accumulated repairs improve new tasks beyond ordinary retries. Moreover, preserving initial base successes follows from the verifier-gated execution policy; it does not establish that library growth preserves earlier repairs when retrieval choices change. Separating reuse and fallback outcomes in existing logs, then replaying a small set of earlier failures before and after library growth, would directly test the paper’s broader continual-learning claim.

I would be happy to increase my score if these weaknesses are addressed.

Questions for authors

Were the textual-memory and adapter conditions given identical failure traces, synthesized examples, retrieval decisions, and fallback budgets? Figure 6 shows different initial success rates across conditions, but paired wins/losses from matched runs or failure subset analysis are missing.

From existing logs, can you separate successes in the three-adapter GAIA stream into newly trained adapter recoveries, reused adapter recoveries, and base fallbacks? Comparing reused adapters with available base retries on the same failures would more directly quantify transfer.

Does adding adapters change retrieval for previously repaired tasks?

2 comments
AuthorsOpenReview ↗

Response to the reviewer 2's review, supported with relevant clarifications and additional experiments and updates to the paper

Responses to the critical weaknesses

1. The experiments do not cleanly establish the benefit of storing lessons in weights rather than retrieving them as text. This distinction is central to the contribution.

We agree that this comparison is central to the contribution. Following the reviewer’s suggestion, we conducted a new paired analysis on 168 shared initial failures across all three benchmarks using Qwen3.6-27B:

BenchmarkShared failuresSelf-Direction full loopIn-contextSkills.md
GAIA711159
Terminal-Bench 2.1441374
Claw-Eval531195
Total168352118

Additionally, with the same retry and fallback budget, the adapter recovers about 14% more failures than in-context reflections and 33% more than Skills.md in our paired comparison. So the observed gain remains without the extra base fallback, while the lessons persist without taking up active context. We thank the reviewer for this suggestion and have added this new paired analysis to the Section 4.1 in our updated paper (changes are highlighted in blue). We have also added a new appendix section K.1, that has the full 168-failure table and how the baselines are matched.

2. Evidence for continual improvement through a growing repair library

We agree that our continual stream result should separate skill reuse from base fallback. Following the reviewer’s suggestion, we ran a new analysis on all 27 recoveries in the three-adapter GAIA stream: 18 come from reused adapters, 9 from base fallbacks, and none from newly trained adapters. Most recoveries therefore use skills learned earlier in the stream, without training a new adapter for those tasks. These new findings have been added to Section 4.4 in our updated paper!

To further check whether reuse helps beyond ordinary retrying, we compared it with eight independent base runs on the same failures. Reused adapters achieve about 25% more recoveries than an average base retry. This shows how reuse gives higher observed recovery even when the base fallback is excluded. These details have also been added to a section 4.4 in the paper.

For earlier repairs, we reload the recorded successful adapter instead of rerunning similarity-based selection. Adding new skills does not change this stored mapping. This keeps earlier repair choices intact while allowing learned skills to help on later tasks, the two properties we want from a continual repair library! We have added the recovery breakdown to Section 4.4 and made this lookup explicit in the Section 3.3 of the updated paper. All changes are highlighted in blue in the updated paper.

Additional Clarifications

1. How are inputs, retrieval and retry budgets controlled

Across all three benchmarks, the in-context baseline uses the exact same failures and verbatim corrective examples used for LoRA training. Skills.md summarizes the same corrective examples and failed trajectories into one file loaded on every retry. In the response to Weakness 1 above, we highlight how Self-Direction enables gains with controlled retry budgets. We thank the reviewer for pointing this out and have updated the draft in Appendix K.1 to clarify that the experiments use the same exact failure and verbatim examples.

2. Does adding adapters change retrieval for previously repaired tasks

For tasks with a recorded successful adapter repair, no. Our failure SQLite database stores the adapter that resolved the task, and later encounters directly reload that adapter. Adding new adapters does not change this stored mapping. This is the protocol used in Table 1, where 76.9% and 67.3% of first-encounter repairs remain effective through encounters 2 and 3. Tasks without a successful adapter repair can still retrieve a better match as the library grows. A growing library of skills therefore keeps known fixes in place, while giving unresolved tasks access to additional skills. We have made this lookup explicit in Section 3.3 in the updated paper.

We thank the reviewer for the suggestions. We hope the two additional experiments controlling for an identical mistake set and analysis separating sources of recovery provide more clear evidence of the benefits of in-weight lessons with a growing library to address your concerns. We’ve updated the draft with the new experiments in Sections 3.1, 4.1 and 4.4 and a detailed breakdown in Appendix K.1. We remain available for further discussion.

Thanks to the authors for their thorough rebuttal. I will raise my score.

Reviewer_NFf4OpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Self-Direction, a framework for persistent LLM agent self-improvement. Upon task failure, the agent generates corrective procedural examples from its failed trajectory, distills them into a lightweight LoRA adapter, and stores the adapter in a skill library for future reuse. Experiments across multiple environments demonstrate improvements.

Critical strengths

Extensive empirical evaluation: The authors conduct comprehensive experiments across diverse agentic benchmarks, covering immediate task recovery, repeated-execution reliability, skill transfer, and several design ablations. These evaluations provide reasonably strong empirical evidence that the proposed framework can improve agent performance across different task settings.

Critical weaknesses

The proposed framework primarily combines established techniques, including failure-driven self-reflection, synthetic supervision, LoRA fine-tuning, and similarity-based skill retrieval. The underlying workflow closely resembles existing agent self-improvement paradigms. Several design choices, such as failure-dependent module selection and embedding-based adapter routing, rely on relatively simple heuristics without sufficient justification or analysis of their necessity.

There is no a clear mechanism to establish the correctness or usefulness of training examples before storing them as skills. This raises concerns about how reliably the approach can improve agents facing persistent failures or complex tasks where correct solutions are difficult to discover. The transfer evaluations mainly involve related tasks within existing benchmarks and relatively small skill libraries.

It remains unclear whether the approach can effectively acquire, select, and reuse skills in genuinely new task distributions or more complex deployment environments.

Questions for authors
  • What is the essential advantage of in-weight adaptation over prompt-based correction?

Since the base model can already generate corrective procedural examples from its failed trajectory, why is it necessary to distill these examples into LoRA adapters rather than directly using them as in-context guidance? Although the paper compares against in-context reflections and Skills.md, could the authors clarify whether these baselines use the same generated corrective examples and comparable computational budgets?

  • How is the reliability of self-generated corrective supervision ensured?

  • The framework selects adapters based on task-description similarity, while the transfer experiments primarily involve related tasks and relatively small skill libraries. How are retrieval thresholds determined when encountering new task distributions?

4 comments
AuthorsOpenReview ↗

Response to the reviewer 1's review, supported with relevant clarifications and additional experiments and updates to the paper

Responses to the critical weaknesses

1. Contribution and justification of the design choices

We agree that the individual components are established. We view this as a strength, since individual components are well-known by the community and easy to implement. Our contribution is the mechanism that brings them together, where the agent turns its own verified failure into a lightweight, routable in-weight repair along with a controlled comparison of where and how such lessons should be stored. Our design is guided by two types of comparisons:

First, in the main paper, we compare Self-Direction against other textual repair baselines. Self-Direction consistently improves performance across all 3 benchmarks compared with simple retrying, reflections in-context and SKills.md with gains ranging from 8.9 to 17 percentage points (Figs. 1, 2, 4).

Second, in Appendix E.1, we compare Self-Direction with 4 alternative ways of storing lessons in-weight. Self-Direction recovers 27.7% of failures versus 16.7% for a single global LoRA trained on all failures. Loading a mismatched adapter reduces recovery to 9.6%, below plain retrying. Self-Direction performs better in both comparisons, supporting the choice of task-specific repairs and relevant adapter selection through robust routing.

We are happy to run a specific additional ablation if the reviewer has one in mind.

2. Reliability of generated supervision and recovery on complex tasks

We assess corrective examples by whether they fix the verified failures by the base agent and whether those repairs persist. Individual examples are not checked against a reference before storage, since reference solutions are unavailable to the agent. Instead, the verifier checks the outcome of the adapter-guided retry. The base weights remain unchanged, and an unsuccessful adapter retry falls back to the base model. We therefore check usefulness through actual task recovery, rather than assume every generated example is correct.

Terminal-Bench tasks require coordinated changes to code, scripts, configuration and running services. For example, the task in Appendix I requires configuring a Git server over SSH, implementing a deployment hook and serving separate branches over HTTPS. After the base agent’s failure, the adapter-guided retry successfully completes this deployment and passes the evaluator’s tests. Across all the hard Terminal-Bench tasks, Self-Direction improves success from 25.6% to 37.8% (Table 6), while GAIA Level 3 improves from 8/26 to 11/26. This shows how Self-Direction enabled gains therefore extend to demanding coding, system and research tasks.

Table 1 further shows that repairs remain effective across future encounters more reliably than base retrying under the same rollout allowance. Separately, Section 4.4 tests reuse on unseen harder tasks and related task categories, while Appendix C.8 evaluates routing with full libraries from other benchmarks. Together, these results support useful recovery on complex tasks, persistent repairs and reuse beyond the task that generated the lesson.

3. Transfer beyond small libraries to new task distributions and related tasks

Appendix C.8 evaluates Self-Direction with full libraries of adapters learned on other benchmarks. On GAIA and Terminal-Bench, task success is preserved for τ ≥ 0.30 even with large libraries of unrelated skills (Fig. 12). Across GAIA, Terminal-Bench and Claw-Eval, a single threshold of 0.55 produces 0/370 cross-domain retrievals (Table 7). These results show that our simple router scales favorably to larger libraries involving hundreds of unrelated adapters while preserving the tested capabilities.

Separately, 28 adapters learned on easier GAIA tasks raise accuracy on unseen harder tasks from 40.7% to 51.2% beating the base agent, without additional training (Section 4.4). This further shows how skills learned through Self-Direction can favorably support recovery on harder, unseen tasks.

Motivated by the reviewer’s question, we have updated our paper to highlight this evidence more clearly in Section 4.3 and the new Discussion section. You can find our updates in the revised paper PDF highlighted in blue. We remain available for further discussion.

Finally, we provide additional clarifications below to support our responses above:

AuthorsOpenReview ↗

Follow-up and further clarifications

Further Clarifications

1. Why distill the examples into weights and how are the baselines matched

Across GAIA, Claw-Eval and Terminal-Bench, the in-context baseline uses the exact same failures and verbatim corrective examples used for LoRA training. Skills.md summarizes the same corrective examples and failed trajectories into one file loaded on every retry. We thank the reviewer for pointing this out and have updated the draft in Appendix K.1 to make clear the experiments use the same exact failure and verbatim examples.

The advantage of in-weight storage is that lessons persist without occupying the active context. On Claw-Eval and Terminal Bench, the adapter retry recovers 17 failures versus 9 for Skills.md, while each adapter occupies approximately 85MB outside the context. In short, in-weight lessons correct more mistakes in this comparison, avoid consuming context, and retain their repairs across future encounters (Table 1).

2. How are retrieval thresholds chosen

The router selects the stored task description with the highest cosine similarity and loads its adapter only above threshold τ. Fig. 12 shows that existing capabilities on GAIA and Terminal-Bench are preserved for τ ≥ 0.30, while lower thresholds can hurt. Across three benchmarks and 185 prior mistakes, τ = 0.55 produces 0/370 cross-domain retrievals (Table 7). This provides an empirical starting point for avoiding unrelated skill retrieval in the tested distributions.

For deployment, we propose starting with this threshold and adjusting it as the library grows. In the GAIA stream, thresholds at the 25th, 50th and 75th percentiles of pairwise task similarities produce libraries of 3, 6 and 12 adapters (App. C.7); these sizes result from the thresholds rather than being predetermined. The threshold controls the trade-off between reusing existing skills and learning new ones. We have made this trade-off clearer in Section 4.3 and the discussion section in the updated paper. We thank the reviewer for the discussion, it has helped us clarify the design choices of our experiments and have made our paper stronger!

AuthorsOpenReview ↗

Request to consider the rebuttal

Dear Reviewer,

As the discussion period nears its end, we wanted to kindly ask whether our rebuttal has adequately addressed your concerns.

Following your feedback, we clarified the baseline comparisons, how generated supervision is evaluated, and how routing handles larger skill libraries in new, complex environments. We also added matched comparisons with in-context reflections and Skills.md, along with analyses of adapter reuse and library utility following the other reviewers’ suggestions. These results and clarifications have been incorporated into the revised paper.

We would be very grateful for any further feedback. Should any point remain unclear or require additional information, we would be glad to respond.

Thank you again for your time and effort in reviewing our paper.

Thanks to the authors for their thorough rebuttal. I will raise my score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.