- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes SPLICE, a framework for knowledge–model co-evolution in LLM agents. The key idea is that different ways of analyzing the same experience may produce knowledge with very different utility, so the authors introduce multiple evolution views and update knowledge only when the resulting utility gains are consistent across views. The proposed common gain combines average gain with a cross-view consistency penalty, and the resulting knowledge is further internalized into the model through GRPO and gated contrastive self-distillation. The framework is evaluated on four domains, including mathematics, coding, spreadsheet manipulation, and web shopping. SPLICE improves both training-free knowledge evolution and knowledge–model co-evolution over several recent baselines, and the evolved knowledge also shows cross-model-size and cross-family transferability.
Critical strengths
-
The problem is practically meaningful. Existing agent knowledge-evolution methods often assume that extracted knowledge is useful, while seemingly reasonable knowledge may not actually improve downstream performance. Using cross-view utility consistency to determine whether a knowledge update is reliable is intuitive and relatively novel.
-
The proposed common gain is simple and directly aligned with the main hypothesis. It considers not only average utility gain but also disagreement across evolution views, and the paper provides a theoretical interpretation for this design.
-
The empirical results are strong and cover four diverse task domains. SPLICE consistently improves over prior methods in both training-free knowledge evolution and knowledge–model co-evolution.
-
The ablation studies directly test the central mechanism. Replacing common gain with mean gain causes a clear performance drop and makes harmful knowledge updates much more frequent, which supports the importance of the consistency term.
-
The cross-model transfer results are convincing. Knowledge evolved with Qwen3.5-9B transfers to other model sizes and different model families, suggesting that the learned knowledge is not entirely model-specific.
Critical weaknesses
-
The theoretical assumptions are not fully aligned with the practical implementation. Proposition 1 assumes that different evolution views are independent and identically distributed, while in practice the views are generated by the same knowledge optimizer from the same experience and may therefore be strongly correlated.
-
The computational overhead is substantial. Each round of knowledge evolution generates multiple candidates and performs task replay, while model evolution additionally requires multiple rollout groups and self-distillation.
-
Knowledge updates are repeatedly accepted or rejected based on a held-out validation set. Such repeated adaptive selection may gradually overfit the validation set.
-
The framework relies on an external LLM as the knowledge optimizer. The quality and diversity of the evolution views therefore depend on the capability of this external model, and robustness across different knowledge optimizers is not sufficiently explored.
Questions for authors
-
Proposition 1 assumes that evolution views are i.i.d., while in practice they are generated by the same knowledge optimizer from the same experience. Could the authors clarify the validity of this assumption and analyze how correlation among views affects common gain?
-
The method relies on an external LLM to generate evolution views and update knowledge. How robust is SPLICE when using weaker knowledge optimizers or optimizers from different model families?
-
The default setting uses three evolution views. How sensitive is performance to the number of views? Do additional views consistently help, or does performance quickly saturate?
-
Knowledge updates are repeatedly accepted or rejected using a held-out validation set. Have the authors observed validation overfitting, and have they tested dynamic validation splits or an independent holdout set?
4 comments
W2: Computational overhead.
We first clarify that multiple rollout groups and self-distillation are not additional costs introduced by SPLICE, but are shared by many existing methods that update the model with RL. SkillRL and ReSkill likewise sample multiple rollouts per task and update their skills during training, and methods that internalize knowledge into model parameters generally also require self-distillation or even heavier training procedures [1,2]. Our measurements agree with this: the training-step cost of SPLICE per parameter update is 0.573 GPU-hours for 9B and 0.272 for 4B, comparable to 0.520 and 0.243 for ReSkill and lower than 0.571 for SkillRL 4B.
The additional cost of SPLICE lies in knowledge evolution. The table below reports the GPU-hours and optimizer tokens per round of the knowledge evolution methods on WebShop. All methods were run on a single 8×H200 node.
| Method | Model | Rollout | Knowledge optimization | Input tokens | Output tokens |
|---|---|---|---|---|---|
| ReasoningBank | 9B | 0.239 | 0.171 | 414K | 8.8K |
| ReasoningBank | 4B | 0.169 | 0.167 | 375K | 8.6K |
| ACE | 9B | 0.707 | 0.677 | 745K | 65.9K |
| ACE | 4B | 0.383 | 0.493 | 536K | 37.0K |
| EvoSkill | 9B | 0.222 | 0.168 | 226K | 5.9K |
| EvoSkill | 4B | 0.155 | 0.250 | 178K | 7.6K |
| SkillOpt | 9B | 0.216 | 0.117 | 493K | 6.3K |
| SkillOpt | 4B | 0.177 | 0.131 | 598K | 6.9K |
| SPLICE | 9B | 0.250 | 0.248 | 764K | 11.3K |
| SPLICE | 4B | 0.170 | 0.272 | 804K | 12.0K |
SPLICE costs more per round than ReasoningBank, EvoSkill, and SkillOpt, while remaining well below ACE. The extra cost comes from the multiple views: each view requires its own optimizer calls, which form the largest component (0.248 GPU-hours for 9B and 0.272 for 4B), and replaying all candidates adds 0.102 and 0.065 GPU-hours. In co-evolution, knowledge evolution runs in parallel with training and therefore does not noticeably slow it down.
This additional cost is worthwhile. SPLICE achieves the highest average accuracy in both the training-free and the training-based groups, and the gains do not come from extra computation alone: the Best branch variant in Figure 4 also generates multiple views and replays their candidates, yet performs clearly worse, indicating that the benefit comes from how common gain exploits utility consistency across views.
[1] Shi Y, Chen Y, Lu Z, et al. Skill1: Unified evolution of skill-augmented agents via reinforcement learning, 2026.
[2] Wu R, Wang X, Mei J, et al. Evolver: Self-evolving llm agents through an experience-driven lifecycle, 2025.
W3 & Q4: Validation overfitting.
In our setup, the validation and test splits are strictly separated: the test split never enters knowledge evolution process. The test results in Table 1 therefore already reflect the generalization of the evolved knowledge. Nevertheless, we agree with the reviewer that repeatedly selecting updates on a fixed validation set could lead to overfitting it. We therefore conducted three additional experiments.
Denser validation and Test curves. On three WebShop 9B knowledge evolution runs, we re-evaluated Test500 with the knowledge retained at certain rounds and compared it with the validation accuracy of the same round. The table reports the average over the three runs (%).
| Round | Val100 | Test500 |
|---|---|---|
| 0 | 7.0 | 10.20 |
| 20 | 51.0 | 56.33 |
| 40 | 52.3 | 56.73 |
| 60 | 56.3 | 59.13 |
| 80 | 56.3 | 59.13 |
The two curves rise together, and we do not observe a divergence where validation keeps improving while Test stalls or drops.
An independent holdout. We held out 400 tasks from the official WebShop dev split that overlap neither the 100 validation tasks nor Test500. They never enter knowledge evolution process, and are used only for post hoc evaluation (accuracy, %).
| Round | 9B Val / Holdout | 4B Val / Holdout |
|---|---|---|
| 0 | 8.0 / 11.25 | 2.0 / 2.50 |
| 20 | 49.0 / 52.50 | 53.0 / 55.25 |
| 40 | 49.0 / 52.50 | 53.0 / 55.25 |
| 60 | 53.0 / 60.00 | 61.0 / 59.00 |
| 80 | 53.0 / 60.00 | 61.0 / 59.00 |
The improvements obtained on the validation set also appear on this holdout, which never took part in any selection.
Dynamic validation splits. We also tested a rotating validation set: the 600 tasks from the dev split are divided into six non-overlapping subsets of 100 tasks, and the active subset rotates every certain rounds. At each rotation the current knowledge is re-evaluated. After the knowledge evolution, the final accuracy is 75.00 on Test500 and 76.25 on holdout400 for 9B, and 65.60 and 60.00 for 4B. Together, these three experiments show no sign that SPLICE overfits a fixed validation set and remains effective when the validation set rotates during evolution.
Overall, the improvements from knowledge evolution also hold on data that never participated in selection, and we do not observe signs of validation overfitting. A more systematic comparison of validation and holdout strategies is left for future work.
W4 & Q2: Knowledge optimizer.
We would like to first clarify that using an LLM to analyze experience and generate knowledge is a common practice in knowledge evolution [1,2,3] rather than a design choice specific to SPLICE. SPLICE also makes no particular assumption about the optimizer. Its contribution lies in selecting evidence through utility consistency across evolution views, and the single-view knowledge generation can be performed by any optimizer.
We nevertheless agree with the reviewer that it is important to examine the robustness of SPLICE to the choice of optimizer. To this end, we replaced the optimizer on WebShop with three weaker models, including models from other families: Claude Haiku 4.5, Qwen3-Next-80B-A3B, and DeepSeek-V3. Under each optimizer, SPLICE, SkillOpt, and EvoSkill each run complete knowledge evolution from empty knowledge with the task model unchanged. The final Test accuracy (%) is:
| Task model | Method | Haiku 4.5 | Qwen3-Next | DeepSeek-V3 |
|---|---|---|---|---|
| 9B | SkillOpt | 7.60 | 12.80 | 11.80 |
| 9B | EvoSkill | 7.80 | 8.60 | 11.20 |
| 9B | SPLICE | 49.60 | 33.00 | 28.00 |
| 4B | SkillOpt | 12.40 | 2.80 | 2.00 |
| 4B | EvoSkill | 1.40 | 5.00 | 2.20 |
| 4B | SPLICE | 49.80 | 39.40 | 21.40 |
SPLICE clearly outperforms the other two methods under every optimizer and for both model sizes, whereas the two baselines nearly fail with the weaker optimizers. The main reason is that weaker optimizers often produce lengthy and overly procedural knowledge. For example, an EvoSkill skill generated by Haiku 4.5 requires multi-step validation before every search, causing many actions to be truncated. SPLICE also produces candidates of mixed quality, but such candidates show consistently negative gains across views during replay and are filtered out by common gain, so only evidence that brings gains under multiple views is retained.
These results suggest that SPLICE remains effective across different optimizers. This is consistent with our motivation: by retaining only evidence whose utility is consistent across views, SPLICE can still extract generalizable knowledge.
[1] Zhao A, Huang D, Xu Q, et al. Expel: Llm agents are experiential learners, 2024
[2] Zhang Q, Hu C, Upasani S, et al. Agentic context engineering: Evolving contexts for self-improving language models, 2026
[3] Yang Y, Gong Z, Huang W, et al. Skillopt: Executive strategy for self-evolving agent skills, 2026
Dear Reviewer,
Thank you again for your thoughtful and constructive review. We have carefully considered your feedback, conducted additional experiments and analyses to address your concerns, and revised the manuscript accordingly.
As the discussion period draws to a close, we would sincerely appreciate your feedback on whether our responses have addressed your main concerns. We would be happy to discuss and clarify any remaining questions.
Thank you for your time and consideration.
Best regards, The Authors
Thank you for the thoughtful and constructive review and for recognizing the idea of common gain and our empirical results. We address your concerns below.
On the i.i.d. assumption. The i.i.d. assumption in Proposition 1 is an idealization, and we agree that it does not strictly hold in practice. Note that the proposition already conditions on the task, the experience, the model, and the current knowledge, so sharing the same experience is part of the conditioning rather than a violation. The remaining dependence comes from the views being generated by the same optimizer, which may give them similar tendencies. The assumption is only needed for the probability bound, which motivates selecting evidence supported by several views. Common gain itself is well defined without it. When the views are dependent, the bound can be optimistic, and we analyze below how such dependence affects common gain in practice.
Moreover, generating the views with the optimizer is only one way to instantiate SPLICE. The views can also be predefined, for example to incorporate human expertise in analyzing experience, and the consistency criterion and common gain apply without change. We have revised the discussion in the paper to clarify these points.
How dependence affects common gain. The more correlated the views are, the smaller the differences among their gains and the smaller the inconsistency penalty, so common gain approaches mean gain. In the extreme case where all views are identical, common gain equals mean gain and the consistency criterion no longer has any effect. Dependence therefore weakens the selection rather than causing the method to select worse evidence. To quantify how much the views actually differ, we measured their gains on the same task. For each round we fix the model and the three candidate knowledge documents, replay each candidate 15 times on every task, average the 15 replays to remove rollout noise, and subtract the reward of the original rollout to obtain the gain Δj of each view on that task. We then report three metrics:
The views disagree substantially on the same task, and a considerable portion of the tasks with a positive mean gain are filtered out as a result. The consistency term therefore remains active rather than degenerating into mean gain, which is consistent with Table 2.
Number of views. We ran complete knowledge evolution on WebShop and LiveMath with N=2,3,4, replaying each candidate once and keeping all other settings unchanged. The final Test accuracy (%) is:
Increasing N from 2 to 3 improves most settings, while increasing it to 4 does not bring consistent further gains, indicating that performance saturates quickly. Since each additional view increases both replay and optimizer calls linearly, the default N=3 is a reasonable trade-off between effectiveness and cost.