- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes Self-Direction, a framework that enables agents to learn from their own failures and retain corrective experience through lightweight LoRA adapters. The experiments examine task performance, repeated-use reliability, and transfer to unseen tasks. I think it is a very good try to compare the memory-style method v.s. parameter-based method for self-reflection.
Critical strengths
- This is a very interesting idea that explores an important direction for continual agent improvement. In particular, the results suggest that parametric memory can be a promising alternative to textual memory for retaining and applying procedural experience.
- The paper also provides useful experiments beyond aggregate task success, including comparisons with textual memory and pooled adapters, repeated-encounter evaluations, and cross-task transfer.
Critical weaknesses
- Failures provide useful signals about what needs improvement, but successful trajectories can also contain reusable procedures, effective tool-use strategies, and robust verification practices. Why not choose the successful trajectories? Or just pairwise learning to train the adaptor?
- The paper considers a capped bank and similarity thresholds, which provide useful initial controls. However, these do not fully address how memory should be maintained over a long interaction history: adapters may become redundant, overlap in functionality, or become outdated. It remains unclear when adapters should be consolidated, updated, or removed.
Questions for authors
- Is failure-triggered learning primarily an efficiency choice, or is there evidence that failure-derived experience produces more useful adapters than successful experience under comparable budgets?
- How does performance change as the adapter bank grows? Can the method identify redundant adapters and retain those with the greatest utility across tasks?
- Would routing based on tool-call sequences, error types, or failure-trajectory representations outperform task-description similarity?
2 comments
Thanks for the author's rebuttal. I encourage the author to include those discussions in the final version. I will keep my positive scores.
Response to the reviewer 3's review, supported with relevant clarifications and additional experiments and updates to the paper
Responses to the critical weaknesses
We thank the reviewer for recognizing the promise of parametric memory and the value of evaluating persistence and transfer beyond aggregate task success.
1. Why focus on failures instead of successful trajectories or pairwise learning
This is a very interesting idea and we would like to thank the reviewer for bringing it up for discussion. The setting that we aim to explore in the current paper is mistake-conditioned, since we focus on examining and fixing how agents recover from failures. For the purposes of this project, we treat failures as the intervention point because they offer learning opportunities beyond the current capabilities of the model and learning something new that the current model lacks. This holds even for a stronger base model, as Qwen3.8-27B fails fewer tasks, yet Self-Direction still adds 16.4 points on GAIA (App. D). It is also an efficiency choice, since adapters are only trained for the tasks that the base model fails (e.g., 101 of 165 GAIA tasks). In short, the failure trigger directs learning toward observed weaknesses and remains useful as the base model improves. We agree with the reviewer that successful trajectories also offer a learning signal and have acknowledged this in the new discussion section of the updated paper. All changes in the updated paper are highlighted in blue.
2. Maintaining the adapter library over a long interaction history
In the GAIA stream, varying the routing threshold produces libraries of 3, 6 and 12 adapters. Final accuracy remains similar, while new training is avoided for 97.1%, 93.5% and 88.7% of failures, respectively (Section 4.4, App. C.7). Cross-benchmark tests with full libraries also preserve existing capabilities on GAIA and Terminal-Bench for τ ≥ 0.30 (App. C.8). These results show that compact libraries can substantially reduce new training requirements while maintaining similar final accuracy in the tested streams.
Motivated by the reviewer’s comment, we conduct additional analysis on the utility of the adapters. In the 12-adapter library, 4 adapters account for all 16 reuse wins, while the other 8 were retrieved 20 times without a win. Since we log the retrievals and wins of every adapter, these counts directly identify redundant adapters, and a simple utility-based policy can follow naturally, e.g., removing adapters that are retrieved repeatedly without a win or consolidating adapters with overlapping source tasks. We agree that this is important for long interaction histories and have highlighted this in the new Discussion Section and Appendix K.2 of the updated paper.
Additional Clarifications
1. Could tool calls, error types or failure trajectories improve routing
This is a great suggestion. We route on task descriptions because they are simple and available for every task, but we agree that the reviewer's alternatives could offer a strong retrieval too. Successful transfers in the GAIA stream share a procedure rather than a topic (Table 14), and same-level and cross-level retrievals succeed at nearly the same rate (14.7% vs. 14.5%, App. H.1). Since retrieval only happens after a failure, the failed trajectory and its tool-call sequence are already available at routing time at no extra cost. Error types could complement the task embedding. Motivated by the reviewer’s question, we have added this to the new discussion section of our paper.