- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper presents an empirical study of how an LLM agent can improve across a stream of tasks in the same environment by retaining experience in context, updating its weights, or combining the two. It asks what weight updates add when an agent already retains its own experience in context, under a common online protocol. It compares raw-trajectory ICL policies, persistent LoRA updates with GRPO or self-distillation (SDPO), and their combinations while scoring each task before adapting to its outcomes. The experiments use Qwen3.5-9B on 109 tasks from four selected environments, with five task orders and 15 configurations including the stateless baseline. The best ICL configuration raises the equal-environment mean success rate from 28.8% to 49.4%; adding GRPO to that policy reaches 51.6%. Further analyses examine database interaction, pass@k on code issues, and the interaction between context retention and training. The authors conclude that context and weights can contribute complementary gains and should be assessed jointly for the particular environment. The paper also describes a modular framework for evaluating these combinations.
Critical strengths
- The comparison is controlled and easy to follow.
- There is credible evidence that experience in context can leave room for useful weight update, mostly given by the financial database results.
- The paper is comprehensive and honest in acknowledging prior work and the claim of contribution.
Critical weaknesses
None
Questions for authors
None
Official Comment by Authors
We thank the reviewer for the careful reading and positive assessment. We appreciate your recognition of the controlled comparison, the evidence that weight updates can complement retained experience, and our positioning relative to prior work. If any concerns remain, we would be happy to provide further clarification or evidence, and would be grateful if resolving them could lead you to consider a higher score.