- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes SkillCome, which combines group contrast optimization with a dual memory mechanism to evolve skills. It samples multiple trajectories per task, contrasts successful and failed trajectories within mixed-outcome groups to provide reliable optimization signals, and accumulates failure and contrast patterns in separate memories to guide skill edits. Experiments on six benchmarks and five models of different scales show that the proposed approach outperforms many recent methods in most settings.
Critical strengths
-
The motivation is clear and intuitive. Contrasting successful and failed rollouts of the same question gives a more informative signal than inferring edits from a single failed trajectory.
-
The experiments are extensive, which cover six benchmarks, five models, and both the self-evolving and strong-guides-weak settings. The paper also includes further analyses of transferability, initialization, agent harnesses, and the optimizer's token cost.
Critical weaknesses
-
The paper does not discuss or compare with the closely related method ReasoningBank [arXiv 2509.25140v2], which also learns from both successful and failed trajectories and uses contrastive signals across multiple rollouts.
-
The comparative results are unclear. The reported average improvement of 5.69% in both the abstract and Section 4.2 cannot be matched to any result in Table 1. A similar issue appears in the introduction.
-
The supporting evidence for group contrast is limited. In Table 2, "SkillCome w/o Memory" underperforms SkillOpt on three of the five benchmarks with DeepSeek-V4-Flash. Moreover, "SkillOpt w/ Memory" achieves a clearly higher average than SkillCome w/o Memory. This suggests that much of the improvement on this model may come from the memory component rather than from group contrast.
-
The ablation study is also incomplete. It does not remove the contrast memory and the failure memory separately, does not include a variant with only contrast reflection, does not ablate the ranking strategy that prioritizes contrast edits, and does not examine the effect of the group size . In addition, all ablations are run only on the two weaker models. I recommend adding these experiments and checking whether the findings hold on stronger models.
Questions for authors
Please see weaknesses.
5 comments
Response to Reviewer b2iP (Part2)
Weakness 2: The comparative results are unclear. The reported average improvement of 5.69% in both the abstract and Section 4.2 cannot be matched to any result in Table 1. A similar issue appears in the introduction.
Response: Thank you for pointing out this issue. We apologize for the lack of clarity in our description. The originally reported value 5.69% refers to the mean improvement of our method over SkillOpt averaged on the five models. Specifically, for each model, we compute the improvement of SkillCome over SkillOpt using the unrounded Avg column in Table 1 of our paper:
| Model | SkillCome | SkillOpt | Improvement |
|---|---|---|---|
| Qwen3.8-Flash-Next | 76.96 | 71.73 | +5.23 |
| DeepSeek-V4-Pro-0813 | 67.27 | 63.53 | +3.74 |
| GLM-5.3 | 72.21 | 64.97 | +7.24 |
| DeepSeek-V4-Flash | 68.97 | 59.96 | +9.01 |
| Qwen3.6-35B-A3B | 70.31 | 67.07 | +3.24 |
The mean improvement is then (5.23 + 3.74 + 7.24 + 9.01 + 3.24) / 5 = 5.69.
Similarly for the introduction, we state: SkillCome enables effective skill evolution and delivers consistent improvements, with average gains up to 19.55% compared to no skill. The 19.55 value refers to the mean improvement of our method over ''No skill'', also computed by the Avg column in Table 1. The mean improvement is then (19.43 + 14.65 + 20.72 + 27.23 + 15.71) / 5 = 19.55.
To eliminate the ambiguity, we have revised the corresponding descriptions in the abstract, introduction, and Section 4.2 to explicitly state the calculation. (Note that following Reviewer wZgb's Weakness 1, we now report LiveMath Non-Meta scores in the revised manuscript. Thus, the corresponding values in the paper are recomputed.)
Weakness 3: The supporting evidence for group contrast is limited. In Table 2, "SkillCome w/o Memory" underperforms SkillOpt on three of the five benchmarks with DeepSeek-V4-Flash. Moreover, "SkillOpt w/ Memory" achieves a clearly higher average than SkillCome w/o Memory. This suggests that much of the improvement on this model may come from the memory component rather than from group contrast.
Response: We would like to clarify that group contrast is effective, and the two components of SkillCome, group contrast reflection and the dual memory, are synergistic.
-
"SkillCome w/o Memory" still outperforms SkillOpt on average. As shown in Table 2 of our paper, although "SkillCome w/o Memory" underperforms SkillOpt on three benchmarks with DeepSeek-V4-Flash, its average score remains higher (61.30 vs. 59.96). Meanwhile, on Qwen3.6-35B-A3B, it surpasses SkillOpt on 5 out of 6 benchmarks.
-
The advantage of "SkillOpt w/ Memory" is model-dependent. We acknowledge that "SkillOpt w/ Memory" achieves a higher average performance than "SkillCome w/o Memory" on DeepSeek-V4-Flash. However, its average is lower on Qwen3.6-35B-A3B. Generally, the relative benefits of the two components vary across models and tasks, and neither alone dominates
-
Contribution of explicit group contrast reflection. Following the suggestion in Reviewer yMx8's
Weakness 1, we conducted an ablation that removes the contrast reflection in SkillCome. "SkillCome w/o Contrast Reflection" reduces performance on all benchmarks, lowering the average performance by 2.15 points. This directly supports the contribution of contrast reflection. -
The two components are complementary. Group contrast provides more informative signals. But under a fixed training budget, it observes fewer distinct questions per step. For example, a batch of 40 trajectories covers 40 different questions under single rollouts, but only 5 different questions under grouped rollouts at n = 8. Consequently, these more informative signals from a small number of questions can be diluted by other optimization signals in the same step. The dual memory system addresses this by accumulating and connecting such signals across steps, so they can guide the optimization direction when sufficient evidence accumulates. Thus, our two components are complementary: group contrast generates precise optimization signals, while memory aggregates them into stable optimization directions.
-
The post-hoc experiment in Figure 1 provides additional evidence. As shown on the right of Figure 1 in our manuscript, we have performed a single round of post-hoc group contrast analysis. This is directly conducted on SkillOpt's existing trajectories (i.e., no additional training rollouts), isolating the effect of group contrast with minimal variables. This post-hoc skill edit alone yields consistent improvements across three datasets, showing that group contrast itself provides valuable optimization signals.
Response to Reviewer b2iP (Part3)
Weakness 4: The ablation study is also incomplete. It does not remove the contrast memory and the failure memory separately, does not include a variant with only contrast reflection, does not ablate the ranking strategy that prioritizes contrast edits, and does not examine the effect of the group size. In addition, all ablations are run only on the two weaker models. I recommend adding these experiments and checking whether the findings hold on stronger models.
Response: We have conducted the requested ablations on Qwen3.6-35B-A3B:
- SkillCome w/o Contrast Memory: Remove only contrast memory while retaining other components of SkillCome.
- SkillCome w/o Failure Memory: Remove only failure memory.
- SkillCome Contrast Reflection Only: Remove both success and failure reflection (and thus failure memory).
- SkillCome w/o Contrast Priority: Remove the ranking preference for contrast edits.
- SkillCome n=4 / n=16: Fix the number of training trajectories per step and vary the group size.
Following your recommendation, we also run the two original ablations and the newly added "SkillCome w/o Contrast Memory", "SkillCome w/o Failure Memory", "SkillCome Contrast Reflection Only", "SkillCome n=4 / n=16", on the stronger Qwen3.8-Flash-Next model. Due to time constraints, we did not include the most time-consuming benchmark ALFWorld. Together with the original ablations in Table 2 of our paper, the results are as follows:
| Model | Method | Spreadsheet | SearchQA | LiveMath (All) | OfficeQA | DocVQA | Avg |
|---|---|---|---|---|---|---|---|
| Qwen3.6-35B-A3B | SkillOpt | 57.65 | 67.36 | 69.81 | 34.63 | 89.77 | 63.84 |
| SkillOpt w/ Memory | 58.63 | 69.07 | 67.92 | 35.14 | 89.91 | 64.13 | |
| SkillCome w/o Memory | 58.72 | 68.57 | 70.99 | 34.46 | 89.84 | 64.52 | |
| SkillCome w/o Contrast Memory | 58.63 | 70.14 | 69.81 | 35.64 | 90.11 | 64.87 | |
| SkillCome w/o Failure Memory | 59.43 | 69.43 | 68.87 | 36.49 | 91.98 | 65.24 | |
| SkillCome Contrast Reflection Only | 60.85 | 70.07 | 75.47 | 38.18 | 89.04 | 66.72 | |
| SkillCome w/o Contrast Priority | 61.30 | 71.29 | 70.75 | 39.86 | 90.84 | 66.81 | |
| SkillCome n=4 | 60.59 | 70.93 | 62.26 | 40.54 | 89.84 | 64.83 | |
| SkillCome n=8 | 61.48 | 71.29 | 71.46 | 40.03 | 91.24 | 67.10 | |
| SkillCome n=16 | 62.28 | 69.29 | 72.17 | 38.51 | 89.57 | 66.36 | |
| Qwen3.8-Flash-Next | SkillOpt | 72.51 | 72.57 | 49.53 | 59.80 | 91.24 | 69.13 |
| SkillOpt w/ Memory | 72.24 | 73.43 | 50.94 | 59.46 | 91.44 | 69.50 | |
| SkillCome w/o Memory | 71.17 | 72.86 | 60.38 | 62.16 | 91.71 | 71.66 | |
| SkillCome w/o Contrast Memory | 72.60 | 74.64 | 62.26 | 61.49 | 91.98 | 72.59 | |
| SkillCome w/o Failure Memory | 73.31 | 73.57 | 61.32 | 60.81 | 92.25 | 72.25 | |
| SkillCome Contrast Reflection Only | 70.64 | 72.43 | 63.21 | 60.47 | 91.44 | 71.64 | |
| SkillCome n=4 | 76.51 | 73.64 | 52.83 | 60.13 | 91.71 | 70.96 | |
| SkillCome n=8 | 75.71 | 74.14 | 63.68 | 63.68 | 92.78 | 74.00 | |
| SkillCome n=16 | 74.73 | 72.43 | 65.09 | 62.83 | 92.51 | 73.52 |
Note that "SkillCome n=8" is our original setting reported in the paper. We can observe that:
-
Both memories contribute. Removing either contrast memory or failure memory reduces average performance on both models, while removing both causes larger drops.
-
Failure and success reflection are useful. Compared to original SkillCome, "SkillCome Contrast Reflection Only" causes clear performance drops on 9 out of 10 model-benchmark pairs. This shows that retaining the success and failure reflection is helpful.
-
The ranking priority has little effect. Removing contrast priority during ranking yields a negligible difference on Qwen3.6-35B-A3B. This is because the ranking rules are simple prompts for the optimizer. The ablation is implemented as a single-sentence change in the prompt. This suggests that the gains do not strongly depend on this ranking heuristic.
-
N=8 gives the best average result. Under the same rollout budget, n = 8 achieves the best average performance on both models. Increasing n to 16 does not further improve the average, showing a trade-off between exploring more trajectories per question and covering more distinct questions. With n=4, each group provides fewer contrast pairs, leading to larger drops.
-
Findings hold on stronger models. Trends observed on Qwen3.6-35B-A3B are generally consistent with the results on Qwen3.8-Flash-Next, confirming the findings are robust on stronger models.
We have incorporated these results and discussions into the Ablation Study section of our paper.
Response to Reviewer b2iP (Part1)
We sincerely thank Reviewer b2iP for your valuable time and constructive feedback. Below, we provide our responses to each point. The manuscript has been revised accordingly, with revisions highlighted in blue.
Response: Thank you for pointing out this important relevant work. We acknowledge that ReasoningBank, particularly its MaTTS-parallel scaling setting, shares the high-level idea of grouped rollout and contrasting successful and failed trajectories. However, the two methods differ fundamentally in their problem setting, the definition and usage of skill, and the optimization process.
ReasoningBank refers to its textual experiences as “memory items,” which serve a similar role to "skill" in our setting. For clarity, we refer to these items as "skill" in the following comparison.
As shown in the differences above, we further elaborate on three key distinctions:
Skill editing vs. skill accumulation. In ReasoningBank, each analysis produces a new skill to an ever-growing pool. It aims to learn more and more new skills for test-time scaling, without further modifying the skill content. In SkillCome, the analysis proposes targeted edits to a given skill. We aim to iteratively refine one single skill as a coherent whole.
Supervision and acceptance. ReasoningBank operates without ground-truth labels, relying on LLM self-judgment for correctness. And new skills are directly accepted without verification. While we use ground-truth labels and validate every candidate skill edit before acceptance.
Cross-step evidence. There is one major difference between ReasoningBank and SkillCome in methodology. SkillCome's dual memory accumulates support for each pattern across different groups and optimization steps. As shown in Figure 3 of our paper, a search pattern can gradually accumulate support from 2 to 14 distinct groups across steps, leading to effecitve skill edits. In contrast, ReasoningBank treats each skill generation as an independent process, with no mechanism to connect observations across different samples.
We have included a discussion of ReasoningBank in "Related Work" of our revised manuscript, as shown below:
ReasoningBank (Ouyang et al., 2026b) represents an early exploration of contrastive analysis based on grouped rollouts, learning growing skill pools for test-time scaling[1] Reasoningbank: Scaling agent self-evolving with reasoning memory, ICLR 2026