- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces AgentBrew, a lifelong knowledge distillation framework designed for weaker LLM agents. It transforms the experience of a strong model into actionable notes for a weaker model. The core approach involves a strong "teacher" model analyzing the "student's" failure trajectories to generate actionable error-correction notes; these notes are iteratively validated through the Student's re-execution in the environment and, once confirmed effective, are stored in long-term memory for retrieval in future tasks. The key idea is that transferred knowledge should not only be correct, but also executable by the weaker student. Experiments on reasoning, coding, and tool-use benchmarks show consistent improvements over other baselines.
Critical strengths
- The problem definition is practical. It accounts for the capability gap between the strong teacher model and weak student model, and achieves promising results by leveraging strong models to guide weak ones.
- Notes introduced is verified in real environments, which relatively validates the effectiveness of the notes.
- The system is easy to maintain and requires no model parameter updates, conferring advantages in API-only settings.
- The work covers mathematics, code generation, tool use, and Terminal-Bench, yielding a relatively broad research scope.
Critical weaknesses
- (Crucial) The method's innovation stems primarily from the combination of existing components; similar methods already exist (e.g., ReasoningBank[1], Agent KB[2], Memento-Skills[3], AutoSkill[4], Trace2Skill[5], etc.), so the novelty of the core technology is somewhat limited. Only compared with baseline methods, with no discussion or comparison against recent similar methods.
- Success on a single task fails to straight establish the causal effect of notes; it cannot, further, demonstrate their cross-task effectiveness.
- Issues such as long-term knowledge accumulation, training overhead, and negative transfer have not yet been sufficiently validated.
- This work does not evaluate the cross-domain effectiveness of notes, which limits the scalability of the this method.
[1] Reasoningbank: Scaling agent self-evolving with reasoning memory.
[2] AGENT KB: A hierarchical memory framework for cross-domain agentic problem solving.
[3] Memento-skills: Let agents design agents.
[4] Autoskill: Experience-driven lifelong learning via skill self-evolution.
[5] Trace2skill: Distill trajectory-local lessons into transferable agent skills.
Questions for authors
- The paper does not sufficiently compare with related methods (see above). Please add discussions on differences from similar approaches and corresponding performance comparisons.
- The student model itself involves sampling randomness; thus, success upon re-execution might not be attributable to the newly added note. Have you evaluate this through no-note retry or with-note retry experiments?
- As the note database grows, will erroneous retrieval, knowledge conflicts, or outdated insights gradually impact efficiency and performance?
- The cross-task analysis in A.4 only reports performance for a single note over 1,200 tasks. Could you provide a more systematic analysis to evaluate the real effectiveness of the AgentBrew system on held-out tasks?