- Rating
- 2
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
This paper investigates continual agent learning with an online learning setting. Specifically, this paper proposes AGSE, which mainly consists of (1) skill-use attribution for more fine-grained credit assignment, (2) evidence-weighted utility updates for robust estimation of the skill utility, (3) guarded lifecycle evolution for selective triggering. Extensive experiments on OfficeQA Pro and SkillFlow demonstrate the effectiveness of the proposed framework.
Critical strengths
- This paper does a strong engineering work.
- Extensive harnesses are evaluated to demonstrate the general applicability of the proposed method.
Critical weaknesses
- The benchmarks (OfficeQA Pro & SkillFlow) are overly small, with merely ~100 tasks, which cannot support the verification of “agent continual learning” in online stream.
- Online agent continual learning (similar to RL) presents much higher variance in the final results than the traditional training-testing setting. Please consider report results with at least 5 repeat runs. Otherwise, the results can lead to biased findings.
- Main results in Table 2 seems not convincing enough. Both baselines (Skill-Creator & SkillFlow-Evo) perform much worse than w/o skill, which makes me question the fairness in the empirical evaluation.
- As shown in Table 6, is a very sensitive hyper-parameter. Setting t=2 or t=8 leads to significant deterioration, which makes the algorithm fragile. Meanwhile, Skill-Creator & SkillFlow-Evo are set with t=4 by default without careful hyper-parameter tuning, which may lead to unfair comparison.
- The technical novelty of this paper is limited. Although the skill-use attribution seems to be well-motivated, evidence-weighted utility updates & guarded lifecycle evolution are just heuristic design choices without any empirical or theoretical insights.
Questions for authors
None
7 comments
Response to Reviewer nXY4(2/3)
W3. Main results in Table 2 seems not convincing enough. Both baselines (Skill-Creator & SkillFlow-Evo) perform much worse than w/o skill, which makes me question the fairness in the empirical evaluation.
A3: We respectfully clarify that the claim that both skill-evolution baselines consistently underperform the w/o Skill baseline does not accurately reflect the results in Table 2. For example, on Qwen Code (Qwen3.6 Plus), Skill-Creator achieves 47.4% on OfficeQA @0, outperforming w/o Skill (44.4%), and 51.2% on OfficeQA @0.001, slightly exceeding w/o Skill (51.1%). On Codex (GPT-5.5), SkillFlow-Evo achieves 71.4% on OfficeQA @0.01, outperforming w/o Skill (70.7%). On SkillFlow, both Skill-Creator (44.31%) and SkillFlow-Evo (43.37%) outperform w/o Skill (42.17%) in completion rate under Qwen Code. More importantly, the effectiveness of skill evolution is not guaranteed merely by creating, modifying, or using skills.
This is also supported by recent empirical studies. SkillsBench (Li et al., 2026) reports that self-generated skills provide no average benefit, with 16 of 84 tasks exhibiting negative performance changes even when skills are provided. Similarly, SkillFlow (Zhang et al., 2026) finds that lifelong skill evolution yields substantially different outcomes across models: while Claude Opus 4.6 improves by 8.43 percentage points, Kimi K2.5 gains only 0.60 points despite a high skill usage rate, and Qwen-Coder-Next even regresses relative to its vanilla setting. These findings demonstrate that skill evolution can introduce ineffective or harmful modifications, and that skill usage alone is not a reliable indicator of utility.
This is precisely the problem AGSE aims to address: rather than assuming that every skill update is beneficial, AGSE identifies how skills are used and whether they contribute effectively to task completion, then uses the resulting evidence to guide selective evolution.
W4. As shown in Table 6, is a very sensitive hyper-parameter. Setting t=2 or t=8 leads to significant deterioration, which makes the algorithm fragile. Meanwhile, Skill-Creator & SkillFlow-Evo are set with t=4 by default without careful hyper-parameter tuning, which may lead to unfair comparison.
A4: We agree that the evolution interval () affects performance, since it determines how frequently accumulated experience is translated into skill updates. To examine whether the baseline configurations were fairly selected, we conducted an additional sensitivity analysis for both Skill-Creator and SkillFlow-Evo on OfficeQA Pro using GPT-5.6-sol.
| Method | t=2 | t=4 | t=8 |
|---|---|---|---|
| SkillFlow-Evo | 60.9 | 64.7 | 61.7 |
| Skill-Creator | 62.4 | 65.4 | 63.9 |
| AGSE | 64.2 | 68.9 | 64.9 |
The results show that (=4) achieves the best performance among the tested configurations for both baselines. Importantly, both baselines exhibit performance changes under different evolution intervals, indicating that this sensitivity is not unique to AGSE. It reflects the general trade-off between accumulating sufficient experience and updating the skill library frequently enough to benefit subsequent tasks.
We will include these additional results and clarify the hyperparameter selection procedure to make the comparison more transparent.
Response to Reviewer nXY4(3/3)
W5. The technical novelty of this paper is limited. Although the skill-use attribution seems to be well-motivated, evidence-weighted utility updates & guarded lifecycle evolution are just heuristic design choices without any empirical or theoretical insights.
A5: We appreciate the reviewer’s comments on the technical novelty. We would like to further clarify the central contributions of AGSE and the importance of the problem it addresses.
As LLM agents increasingly operate over streams of related tasks, enabling them to continuously accumulate, retain, and reuse experience becomes an important direction toward more capable and adaptive agents. Rather than treating each task as an isolated problem, continual learning agents should be able to improve their reusable capabilities based on previous interactions. However, an important challenge is how to turn task execution experience into meaningful and actionable updates to the agent's knowledge and skills.
AGSE addresses this challenge by introducing attribution-guided skill evolution, which gives skill self-evolution an explicit direction rather than relying on undirected reflection or task-level success/failure signals. Its central contribution is a five-dimensional skill-use attribution mechanism that distinguishes Activation, Adherence, Correctness, Coverage, and Precision. These dimensions identify different aspects of skill usage and provide actionable evidence about why a skill may fail to contribute effectively to task completion. For example, a skill that is not activated requires a different response from one that is activated but contains incorrect guidance or insufficient coverage.
Building on this attribution mechanism, AGSE integrates two complementary components:
| Component | Role in continual skill evolution |
|---|---|
| Evidence-weighted utility updates | Accumulates dimension-specific feedback across tasks to track the evolving utility of each skill, reducing reliance on individual task outcomes. |
| Guarded lifecycle evolution | Uses the attributed evidence and utility changes to guide selective skill preservation, repair, expansion, scope adjustment, and other lifecycle operations. |
The key contribution is therefore not simply to make agents revise their skills, but to enable them to understand how their existing skills are used, identify what needs to improve, and evolve skills in a corresponding direction over time. This provides a structured mechanism for connecting task-level experience with skill-level improvement in online continual agent learning.
Response to Reviewer nXY4(1/3)
We thank the reviewer for the thoughtful and constructive feedback. We respectfully clarify several concerns regarding benchmark scale, experimental reliability, baseline fairness, hyperparameter sensitivity, and technical novelty. To address these concerns, we conducted additional experiments on OfficeQA Pro using GPT-5.6-Sol due to resource constraints, including repeated runs, reversed task-order evaluation, baseline hyperparameter sensitivity analysis.
A1: We agree that evaluating online continual agent learning with larger and more diverse task streams is important. However, the scale of agent benchmarks should also be considered in light of the time cost and complexity of individual tasks.
In particular, each OfficeQA Pro task requires approximately 7–8 minutes of agent execution and consumes around 1M tokens on average. The benchmark contains 133 questions grounded in approximately 89K pages of Treasury Bulletins and over 26M numerical values, requiring substantial document retrieval, numerical reasoning, and multi-step problem solving. Consequently, scaling such evaluations to thousands of tasks incurs substantial time costs, especially when each task must be executed sequentially on continual learning settings to evaluate the evolution and reuse of skills.
We selected OfficeQA Pro and SkillFlow as our evaluation benchmarks because they involve challenging agent tasks that require substantial problem-solving capabilities and provide opportunities for cross-task skill reuse. In our evaluation, the agent incrementally updates a shared skill library after completing each task, allowing subsequent tasks to benefit from previously accumulated experience. We acknowledge that the number of tasks in these benchmarks limits the extent to which our findings can be generalized to substantially longer task streams, and will clarify this limitation in the paper.
A2: We would first like to clarify why repeated experiments were not included in the original submission. Our evaluation involves long-horizon LLM agent interactions, where each task requires multiple rounds of tool execution, reasoning, and skill utilization. In particular, a single OfficeQA Pro task typically takes approximately 7–8 minutes to complete and consumes around 1M tokens on average. Consequently, evaluating the entire benchmark already incurs substantial time costs. Nevertheless, we agree that repeated experiments are important for assessing the robustness of our method. We conducted three additional repeated runs for all methods on OfficeQA Pro. All additional experiments reported in this response were conducted using GPT-5.6-sol. The results are summarized below.
AGSE consistently outperforms the w/o Skill baseline across all three runs, with an average improvement of 6.0 percentage points. These results provide additional evidence that the observed gains are not attributable to a single run.
We also conducted a task-order robustness check by reversing the original task order. AGSE obtains 67.8% accuracy under the reversed order, compared with 68.9% under the original order. The relatively small difference suggests that the performance is not strongly dependent on the particular task order evaluated in this additional check.