acceptodds
Under review as a conference paper at ICLR 2027

SkillCome: Group Contrast Skill Optimization with Dual Memory

Abstract

Skill evolution improves the capabilities of large language models by analyzing trajectories generated under a given skill and modifying the skill accordingly. Existing approaches typically generate a single trajectory per question. However, this provides insufficient optimization signals since it requires inferring effective skill edits from a solitary path. It is difficult to pinpoint which actions caused the failure in a failed trajectory, or to determine which actions in a successful one should be incorporated into the skill. Furthermore, they rely on a local batch of trajectories for analysis, making the optimization direction susceptible to noisy evidence. To address these, we propose SkillCome, a Skill-evolution method based on group Contrast optimization with dual memory. For each question, SkillCome generates trajectories and performs group contrast analysis to precisely identify key behavioral divergences between successful and failed trajectories, offering reliable optimization signals. The dual memory system further accumulates evidence from historical steps to track patterns shared across different groups, leading to more generalized optimization directions. Together, SkillCome builds a systematic optimization process that transforms experience from observed successful trajectories into reusable skills. Extensive experiments on six benchmarks spanning question answering, reasoning, and agentic tasks demonstrate the effectiveness of our method. SkillCome consistently outperforms baselines across five models of varying families and scales, with gains up to **+5.69** points.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Skill Evolution, Agent Skills

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 2 reviews, above 27% of the others.
Reviewer_b2iPOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes SkillCome, which combines group contrast optimization with a dual memory mechanism to evolve skills. It samples multiple trajectories per task, contrasts successful and failed trajectories within mixed-outcome groups to provide reliable optimization signals, and accumulates failure and contrast patterns in separate memories to guide skill edits. Experiments on six benchmarks and five models of different scales show that the proposed approach outperforms many recent methods in most settings.

Critical strengths
  • The motivation is clear and intuitive. Contrasting successful and failed rollouts of the same question gives a more informative signal than inferring edits from a single failed trajectory.

  • The experiments are extensive, which cover six benchmarks, five models, and both the self-evolving and strong-guides-weak settings. The paper also includes further analyses of transferability, initialization, agent harnesses, and the optimizer's token cost.

Critical weaknesses
  • The paper does not discuss or compare with the closely related method ReasoningBank [arXiv 2509.25140v2], which also learns from both successful and failed trajectories and uses contrastive signals across multiple rollouts.

  • The comparative results are unclear. The reported average improvement of 5.69% in both the abstract and Section 4.2 cannot be matched to any result in Table 1. A similar issue appears in the introduction.

  • The supporting evidence for group contrast is limited. In Table 2, "SkillCome w/o Memory" underperforms SkillOpt on three of the five benchmarks with DeepSeek-V4-Flash. Moreover, "SkillOpt w/ Memory" achieves a clearly higher average than SkillCome w/o Memory. This suggests that much of the improvement on this model may come from the memory component rather than from group contrast.

  • The ablation study is also incomplete. It does not remove the contrast memory and the failure memory separately, does not include a variant with only contrast reflection, does not ablate the ranking strategy that prioritizes contrast edits, and does not examine the effect of the group size nn. In addition, all ablations are run only on the two weaker models. I recommend adding these experiments and checking whether the findings hold on stronger models.

Questions for authors

Please see weaknesses.

5 comments
AuthorsOpenReview ↗

Response to Reviewer b2iP (Part1)

We sincerely thank Reviewer b2iP for your valuable time and constructive feedback. Below, we provide our responses to each point. The manuscript has been revised accordingly, with revisions highlighted in blue.

Weakness 1: The paper does not discuss or compare with the closely related method ReasoningBank [arXiv 2509.25140v2], which also learns from both successful and failed trajectories and uses contrastive signals across multiple rollouts.

Response: Thank you for pointing out this important relevant work. We acknowledge that ReasoningBank, particularly its MaTTS-parallel scaling setting, shares the high-level idea of grouped rollout and contrasting successful and failed trajectories. However, the two methods differ fundamentally in their problem setting, the definition and usage of skill, and the optimization process.

ReasoningBank refers to its textual experiences as “memory items,” which serve a similar role to "skill" in our setting. For clarity, we refer to these items as "skill" in the following comparison.

DimensionReasoningBankSkillCome
Problem settingTest-time scaling; no train/val/test separationTraining-time skill optimization with explicit train/val/test splits
Skill formA growing pool of independent skillsA single unified skill document
Skill editNo; Once generated, each skill remains fixedYes, the skill document is iteratively edited
Skill inputRetrieve a relevant skill per sample; input skills may vary across samplesLoad the fixed optimized skill; no retrieval
Output per groupNew skillsSkill edits (add/replace/delete) to the existing skill
Supervision signalLLM Judged by the agent itselfGround-truth labels
Validation gateNone; new skills are directly appended to the poolSkill edits are accepted/rejected based on validation score
Cross-step accumulationNone; each extraction is independent; no explicit tracking of recurring patternsDual memory accumulates cross-step evidence

As shown in the differences above, we further elaborate on three key distinctions:

  1. Skill editing vs. skill accumulation. In ReasoningBank, each analysis produces a new skill to an ever-growing pool. It aims to learn more and more new skills for test-time scaling, without further modifying the skill content. In SkillCome, the analysis proposes targeted edits to a given skill. We aim to iteratively refine one single skill as a coherent whole.

  2. Supervision and acceptance. ReasoningBank operates without ground-truth labels, relying on LLM self-judgment for correctness. And new skills are directly accepted without verification. While we use ground-truth labels and validate every candidate skill edit before acceptance.

  3. Cross-step evidence. There is one major difference between ReasoningBank and SkillCome in methodology. SkillCome's dual memory accumulates support for each pattern across different groups and optimization steps. As shown in Figure 3 of our paper, a search pattern can gradually accumulate support from 2 to 14 distinct groups across steps, leading to effecitve skill edits. In contrast, ReasoningBank treats each skill generation as an independent process, with no mechanism to connect observations across different samples.

We have included a discussion of ReasoningBank in "Related Work" of our revised manuscript, as shown below:

ReasoningBank (Ouyang et al., 2026b) represents an early exploration of contrastive analysis based on grouped rollouts, learning growing skill pools for test-time scaling

[1] Reasoningbank: Scaling agent self-evolving with reasoning memory, ICLR 2026

AuthorsOpenReview ↗

Response to Reviewer b2iP (Part2)

Weakness 2: The comparative results are unclear. The reported average improvement of 5.69% in both the abstract and Section 4.2 cannot be matched to any result in Table 1. A similar issue appears in the introduction.

Response: Thank you for pointing out this issue. We apologize for the lack of clarity in our description. The originally reported value 5.69% refers to the mean improvement of our method over SkillOpt averaged on the five models. Specifically, for each model, we compute the improvement of SkillCome over SkillOpt using the unrounded Avg column in Table 1 of our paper:

ModelSkillComeSkillOptImprovement
Qwen3.8-Flash-Next76.9671.73+5.23
DeepSeek-V4-Pro-081367.2763.53+3.74
GLM-5.372.2164.97+7.24
DeepSeek-V4-Flash68.9759.96+9.01
Qwen3.6-35B-A3B70.3167.07+3.24

The mean improvement is then (5.23 + 3.74 + 7.24 + 9.01 + 3.24) / 5 = 5.69.

Similarly for the introduction, we state: SkillCome enables effective skill evolution and delivers consistent improvements, with average gains up to 19.55% compared to no skill. The 19.55 value refers to the mean improvement of our method over ''No skill'', also computed by the Avg column in Table 1. The mean improvement is then (19.43 + 14.65 + 20.72 + 27.23 + 15.71) / 5 = 19.55.

To eliminate the ambiguity, we have revised the corresponding descriptions in the abstract, introduction, and Section 4.2 to explicitly state the calculation. (Note that following Reviewer wZgb's Weakness 1, we now report LiveMath Non-Meta scores in the revised manuscript. Thus, the corresponding values in the paper are recomputed.)

Weakness 3: The supporting evidence for group contrast is limited. In Table 2, "SkillCome w/o Memory" underperforms SkillOpt on three of the five benchmarks with DeepSeek-V4-Flash. Moreover, "SkillOpt w/ Memory" achieves a clearly higher average than SkillCome w/o Memory. This suggests that much of the improvement on this model may come from the memory component rather than from group contrast.

Response: We would like to clarify that group contrast is effective, and the two components of SkillCome, group contrast reflection and the dual memory, are synergistic.

  1. "SkillCome w/o Memory" still outperforms SkillOpt on average. As shown in Table 2 of our paper, although "SkillCome w/o Memory" underperforms SkillOpt on three benchmarks with DeepSeek-V4-Flash, its average score remains higher (61.30 vs. 59.96). Meanwhile, on Qwen3.6-35B-A3B, it surpasses SkillOpt on 5 out of 6 benchmarks.

  2. The advantage of "SkillOpt w/ Memory" is model-dependent. We acknowledge that "SkillOpt w/ Memory" achieves a higher average performance than "SkillCome w/o Memory" on DeepSeek-V4-Flash. However, its average is lower on Qwen3.6-35B-A3B. Generally, the relative benefits of the two components vary across models and tasks, and neither alone dominates

  3. Contribution of explicit group contrast reflection. Following the suggestion in Reviewer yMx8's Weakness 1, we conducted an ablation that removes the contrast reflection in SkillCome. "SkillCome w/o Contrast Reflection" reduces performance on all benchmarks, lowering the average performance by 2.15 points. This directly supports the contribution of contrast reflection.

  4. The two components are complementary. Group contrast provides more informative signals. But under a fixed training budget, it observes fewer distinct questions per step. For example, a batch of 40 trajectories covers 40 different questions under single rollouts, but only 5 different questions under grouped rollouts at n = 8. Consequently, these more informative signals from a small number of questions can be diluted by other optimization signals in the same step. The dual memory system addresses this by accumulating and connecting such signals across steps, so they can guide the optimization direction when sufficient evidence accumulates. Thus, our two components are complementary: group contrast generates precise optimization signals, while memory aggregates them into stable optimization directions.

  5. The post-hoc experiment in Figure 1 provides additional evidence. As shown on the right of Figure 1 in our manuscript, we have performed a single round of post-hoc group contrast analysis. This is directly conducted on SkillOpt's existing trajectories (i.e., no additional training rollouts), isolating the effect of group contrast with minimal variables. This post-hoc skill edit alone yields consistent improvements across three datasets, showing that group contrast itself provides valuable optimization signals.

AuthorsOpenReview ↗

Response to Reviewer b2iP (Part3)

Weakness 4: The ablation study is also incomplete. It does not remove the contrast memory and the failure memory separately, does not include a variant with only contrast reflection, does not ablate the ranking strategy that prioritizes contrast edits, and does not examine the effect of the group size. In addition, all ablations are run only on the two weaker models. I recommend adding these experiments and checking whether the findings hold on stronger models.

Response: We have conducted the requested ablations on Qwen3.6-35B-A3B:

  • SkillCome w/o Contrast Memory: Remove only contrast memory while retaining other components of SkillCome.
  • SkillCome w/o Failure Memory: Remove only failure memory.
  • SkillCome Contrast Reflection Only: Remove both success and failure reflection (and thus failure memory).
  • SkillCome w/o Contrast Priority: Remove the ranking preference for contrast edits.
  • SkillCome n=4 / n=16: Fix the number of training trajectories per step and vary the group size.

Following your recommendation, we also run the two original ablations and the newly added "SkillCome w/o Contrast Memory", "SkillCome w/o Failure Memory", "SkillCome Contrast Reflection Only", "SkillCome n=4 / n=16", on the stronger Qwen3.8-Flash-Next model. Due to time constraints, we did not include the most time-consuming benchmark ALFWorld. Together with the original ablations in Table 2 of our paper, the results are as follows:

ModelMethodSpreadsheetSearchQALiveMath (All)OfficeQADocVQAAvg
Qwen3.6-35B-A3BSkillOpt57.6567.3669.8134.6389.7763.84
SkillOpt w/ Memory58.6369.0767.9235.1489.9164.13
SkillCome w/o Memory58.7268.5770.9934.4689.8464.52
SkillCome w/o Contrast Memory58.6370.1469.8135.6490.1164.87
SkillCome w/o Failure Memory59.4369.4368.8736.4991.9865.24
SkillCome Contrast Reflection Only60.8570.0775.4738.1889.0466.72
SkillCome w/o Contrast Priority61.3071.2970.7539.8690.8466.81
SkillCome n=460.5970.9362.2640.5489.8464.83
SkillCome n=861.4871.2971.4640.0391.2467.10
SkillCome n=1662.2869.2972.1738.5189.5766.36
Qwen3.8-Flash-NextSkillOpt72.5172.5749.5359.8091.2469.13
SkillOpt w/ Memory72.2473.4350.9459.4691.4469.50
SkillCome w/o Memory71.1772.8660.3862.1691.7171.66
SkillCome w/o Contrast Memory72.6074.6462.2661.4991.9872.59
SkillCome w/o Failure Memory73.3173.5761.3260.8192.2572.25
SkillCome Contrast Reflection Only70.6472.4363.2160.4791.4471.64
SkillCome n=476.5173.6452.8360.1391.7170.96
SkillCome n=875.7174.1463.6863.6892.7874.00
SkillCome n=1674.7372.4365.0962.8392.5173.52

Note that "SkillCome n=8" is our original setting reported in the paper. We can observe that:

  1. Both memories contribute. Removing either contrast memory or failure memory reduces average performance on both models, while removing both causes larger drops.

  2. Failure and success reflection are useful. Compared to original SkillCome, "SkillCome Contrast Reflection Only" causes clear performance drops on 9 out of 10 model-benchmark pairs. This shows that retaining the success and failure reflection is helpful.

  3. The ranking priority has little effect. Removing contrast priority during ranking yields a negligible difference on Qwen3.6-35B-A3B. This is because the ranking rules are simple prompts for the optimizer. The ablation is implemented as a single-sentence change in the prompt. This suggests that the gains do not strongly depend on this ranking heuristic.

  4. N=8 gives the best average result. Under the same rollout budget, n = 8 achieves the best average performance on both models. Increasing n to 16 does not further improve the average, showing a trade-off between exploring more trajectories per question and covering more distinct questions. With n=4, each group provides fewer contrast pairs, leading to larger drops.

  5. Findings hold on stronger models. Trends observed on Qwen3.6-35B-A3B are generally consistent with the results on Qwen3.8-Flash-Next, confirming the findings are robust on stronger models.

We have incorporated these results and discussions into the Ablation Study section of our paper.

Reviewer_yMx8OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

SkillCome optimizes an external skill document while keeping the model weights frozen. It samples multiple trajectories for each training question and compares successful and failed attempts to propose skill edits. A failure memory and a contrast memory retain patterns across optimization steps, including the questions supporting each pattern. Proposed edits are merged, ranked, and accepted when they improve validation performance. The paper evaluates the approach on six benchmarks with five target models and includes component ablations, transfer experiments, and optimizer token-cost comparisons.

Critical strengths
  1. Comparing successful and failed attempts at the same question gives the optimizer a concrete alternative to reasoning from failures alone. Combining this with historical evidence is a sensible design: a potentially useful lesson can survive a rejected edit and accumulate support from other questions. The ALFWorld example illustrates this process clearly.

  2. The evaluation is reasonably broad, covering five target models and six benchmarks. SkillCome achieves the best result in 26 of the 27 supported model–benchmark pairs. The paper also examines transfer and reports lower optimizer token usage than SkillOpt in the configurations tested in Appendix C.9. These experiments provide evidence of practical value beyond a single favorable setting.

Critical weaknesses
  1. The current ablations do not establish which parts of the proposed design are necessary. Relative to SkillOpt, SkillCome changes the allocation of rollouts across questions, adds contrast-reflection calls, introduces contrast memory, and changes how edits are prioritized. “SkillOpt w/ Memory” adds only failure memory, whereas full SkillCome uses both memories. Consequently, the comparison does not isolate the benefit of explicit same-question contrast or of maintaining two separate memories. The no-memory variant is also less consistent than the discussion suggests: for DeepSeek-V4-Flash, it underperforms SkillOpt on SpreadsheetBench, SearchQA, and LiveMath. A control using the same grouped trajectories and memory budget, but without explicit contrast reflection, would make the main contribution more convincing. Comparing the dual memory with a single combined memory would address the second design claim.

  2. Some of the largest improvements require stronger controls before they can support the paper’s interpretation of skill learning. Appendix C.1 shows that DeepSeek-V4-Flash learns to select a particular meta-option on LiveMath: accuracy on that subset rises from 10.33% to 96.74% relative to the initial skill, while Non-Meta accuracy rises only from 54.58% to 57.08%. The authors appropriately acknowledge this shortcut, but corresponding subset results for the competing methods are missing. On OfficeQA, the unusually large gain for the same model is associated with correcting answer delimiters. This is useful, but a simple formatting instruction or normalized answer extraction could explain part of the advantage without requiring the full method. Without these controls, it is difficult to assess how much of the reported improvement reflects better task-solving procedures rather than adaptation to evaluation conventions.

Questions for authors
  1. With the same grouped rollouts, optimizer model, and memory budget, how much does explicit contrast reflection improve over ordinary success/failure reflection? Does separating failure and contrast memories outperform storing the same evidence in one combined memory?

  2. Could you report LiveMath Meta/Non-Meta results for SkillOpt and SkillOpt-Lite, alongside SkillCome? For OfficeQA, how much of the improvement remains after giving all methods the same explicit answer-format instruction or normalizing the answer delimiters during evaluation?

  3. Do the four independent runs repeat the entire skill-evolution process, or only evaluate a fixed final skill? Please report variability across independent optimization runs for representative settings, since the rollout evidence and validation-based acceptance decisions can both affect the final skill.

  4. Appendix B.2 states that the rollout budget is matched, but the allocation is not fully specified. Please give the numbers of optimization steps, distinct questions per step, training trajectories, and validation executions for each method. How are memory size and skill-document length controlled as optimization proceeds?

5 comments
AuthorsOpenReview ↗

Response to Reviewer yMx8 (Part1)

We sincerely thank Reviewer yMx8 for your valuable time and constructive feedback. Below, we provide our responses to each point. The manuscript has been revised accordingly, with revisions highlighted in blue.

Weakness 1: The current ablations do not establish which parts of the proposed design are necessary. Relative to SkillOpt, SkillCome changes the allocation of rollouts across questions, adds contrast-reflection calls, introduces contrast memory, and changes how edits are prioritized. “SkillOpt w/ Memory” adds only failure memory, whereas full SkillCome uses both memories. Consequently, the comparison does not isolate the benefit of explicit same-question contrast or of maintaining two separate memories. The no-memory variant is also less consistent than the discussion suggests: for DeepSeek-V4-Flash, it underperforms SkillOpt on SpreadsheetBench, SearchQA, and LiveMath. A control using the same grouped trajectories and memory budget, but without explicit contrast reflection, would make the main contribution more convincing. Comparing the dual memory with a single combined memory would address the second design claim.

& Question 1: With the same grouped rollouts, optimizer model, and memory budget, how much does explicit contrast reflection improve over ordinary success/failure reflection? Does separating failure and contrast memories outperform storing the same evidence in one combined memory?

Response: Thank you for these suggestions. We agree that the original ablations did not fully isolate each component's contribution. We also acknowledge that “SkillCome w/o Memory” does not consistently outperform SkillOpt on DeepSeek-V4-Flash, and have revised the discussion. Following your recommendation, we have conducted two additional controlled experiments on Qwen3.6-35B-A3B.

  • "SkillCome w/o Contrast Reflection" only removes the contrast reflection in SkillCome, and keeps the same grouped rollouts & memory budget as SkillCome.

  • "SkillCome w/ Combined Memory" merges failure memory and contrast memory into a single combined memory. Thus, both contrast reflection and failure reflection read and write this same combined memory. The total memory budget remains unchanged.

The results are as follows:

MethodSpreadsheetSearchQALiveMathALFWorldOfficeQADocVQAAvg
SkillOpt57.6567.3669.8183.2134.6389.7767.07
SkillOpt w/ Memory58.6369.0767.9281.1635.1489.9166.97
SkillCome w/o Memory58.7268.5770.9983.4034.4689.8467.66
SkillCome w/o Contrast Reflection59.0769.7168.8782.8437.8490.6468.16
SkillCome61.4871.2971.4686.3840.0391.2470.31
SkillCome w/ Combined Memory57.3070.8666.0484.3343.2490.6468.74

The two new ablations in the table are in bold. We can observe that removing contrast reflection leads to consistent degradation across all six benchmarks, lowering the average by 2.15 points. This directly supports the contribution of the contrast reflection in our method.

"SkillCome w/ Combined Memory" also degrades performance on 5 of the 6 benchmarks, lowering the average by 1.57 points. This validates our dual-memory design. Although the merged memory remains effective, it makes it harder for the optimizer to distinguish failure-derived and contrast-derived patterns.

We have revised the Ablation Study section in our paper to incorporate these discussions.

AuthorsOpenReview ↗

Response to Reviewer yMx8 (Part2)

Weakness 2: Some of the largest improvements require stronger controls before they can support the paper’s interpretation of skill learning. Appendix C.1 shows that DeepSeek-V4-Flash learns to select a particular meta-option on LiveMath: accuracy on that subset rises from 10.33% to 96.74% relative to the initial skill, while Non-Meta accuracy rises only from 54.58% to 57.08%. The authors appropriately acknowledge this shortcut, but corresponding subset results for the competing methods are missing. On OfficeQA, the unusually large gain for the same model is associated with correcting answer delimiters. This is useful, but a simple formatting instruction or normalized answer extraction could explain part of the advantage without requiring the full method. Without these controls, it is difficult to assess how much of the reported improvement reflects better task-solving procedures rather than adaptation to evaluation conventions.

& Question 2: Could you report LiveMath Meta/Non-Meta results for SkillOpt and SkillOpt-Lite, alongside SkillCome? For OfficeQA, how much of the improvement remains after giving all methods the same explicit answer-format instruction or normalizing the answer delimiters during evaluation?

Response: We agree that these controls help to distinguish task-solving skills rather than adaptation to evaluation conventions. For LiveMath, we have expanded Table 4 in Appendix C.1 to include the complete All / Meta / Non-Meta results for all methods:

ModelSkillAllMetaNon-Meta
GLM-5.3No skill36.0814.6752.50
Init skill38.4416.3055.42
Trace2Skill36.7911.9655.83
SkillOpt-Lite65.0995.6541.67
SkillOpt45.0536.4151.67
SkillCome67.9271.7465.00
DeepSeek-V4-FlashNo skill31.604.3552.50
Init skill35.3810.3354.58
Trace2Skill33.2510.3350.83
SkillOpt-Lite33.736.5254.58
SkillOpt68.6396.7447.08
SkillCome74.2996.7457.08
Qwen3.8-Flash-NextNo skill33.7315.7647.50
Init skill35.1419.5747.08
Trace2Skill31.6011.9446.67
SkillOpt-Lite41.7533.7047.92
SkillOpt49.5352.7247.08
SkillCome63.6883.1548.75
DeepSeek-V4-Pro-0813No skill28.074.3546.25
Init skill31.3713.5945.00
Trace2Skill33.2515.2247.08
SkillOpt-Lite32.0810.8848.33
SkillOpt48.3561.9637.92
SkillCome53.3058.7049.16
Qwen3.6-35B-A3BNo skill26.659.2440.00
Init skill30.196.5248.33
Trace2Skill31.609.2448.75
SkillOpt-Lite29.013.8048.33
SkillOpt69.8193.4851.67
SkillCome71.4695.1153.33

As shown, the Non-Meta subset shows smaller gains from skill evolution methods. Nevertheless, SkillCome still achieves the best Non-Meta performance across all five models, confirming that our improvements are not solely from the shortcut.

To isolate the impact of the DSML format issue on DeepSeek-V4-Flash, we followed your suggestion to perform normalized answer extraction. Specifically, the normalized extractor also accepts <|DSML|answer>...</|DSML|answer> delimiters. The results are as follows:

SkillOriginal resultsNormalized results
No skill5.4140.54
Init skill11.6643.92
Trace2Skill12.8441.89
SkillOpt-Lite24.6645.95
SkillOpt26.0147.30
SkillCome50.5150.51

When excluding the impact of the DSML format issue, the baselines recover performance. Nevertheless, SkillCome still achieves the highest score, outperforming the second-best SkillOpt by 3.21 points (50.51 vs. 47.30). This shows that our method's improvement on OfficeQA is not solely from correcting answer delimiters, but also learning better task-solving procedures.

AuthorsOpenReview ↗

Response to Reviewer yMx8 (Part3)

Question 3: Do the four independent runs repeat the entire skill-evolution process, or only evaluate a fixed final skill? Please report variability across independent optimization runs for representative settings, since the rollout evidence and validation-based acceptance decisions can both affect the final skill.

Response: The four runs in our manuscript refer to evaluating the final skill four times on the test set. Following your suggestion, we conducted three independent runs of the entire skill-evolution process on Qwen3.6-35B-A3B:

RunSpreadsheetSearchQALiveMathALFWorldOfficeQADocVQA
Run 161.4871.2971.4686.3840.0391.24
Run 264.4171.0772.8885.8237.8392.51
Run 361.3071.2972.1788.8042.5790.91
Mean62.4071.2272.1787.0040.1491.55

Run 1 in the above table corresponds to the results reported in our paper. These results show that the overall performance is relatively stable across independent optimization runs.

Question 4: Appendix B.2 states that the rollout budget is matched, but the allocation is not fully specified. Please give the numbers of optimization steps, distinct questions per step, training trajectories, and validation executions for each method. How are memory size and skill-document length controlled as optimization proceeds?

Response: Thank you for pointing this out. As described in Implementation Details of our paper, training typically runs for 10 optimization steps. The per-step rollout budget is 80 trajectories for datasets with larger training sets and 40 for smaller ones (the split is specified in Appendix B.1). We provide the requested details below:

MethodOptimization stepsValidation executionsTraining trajectories per stepDistinct questions per stepRollout n
SkillOpt-Lite101180 / 4080 / 401
SkillOpt101180 / 4080 / 401
SkillCome101180 / 4010 / 58

The 11 validation executions include evaluating the initial skill before training, plus the candidate skill after each of the 10 optimization steps.

For memory size, we set a maximum capacity of 6,000 characters for each memory. In practice, we found the memory content fits within this budget without truncation. We do not set a strict limit on the skill-document length. For your reference, we provide the final skill-document lengths on Qwen3.8-Flash-Next:

DatasetSpreadsheetSearchQAOfficeQALiveMathALFWorldDocVQA
Skill length (characters)14,29716,6066,3496,41114,0835,331

We have revised Appendix B.2 of our paper to incorporate these details.

Comments on the paper

3 comments
AuthorsOpenReview ↗

Response to Reviewer wZgb (Part1)

We sincerely thank Reviewer wZgb for your valuable time and constructive feedback. Below, we provide our responses to each point. The manuscript has been revised accordingly, with revisions highlighted in blue.

Weakness 1: Headline gains on LiveMath include a benchmark shortcut. The authors themselves show that the large improvements of the weaker models mainly come from the meta-option shortcut (Flash: Meta 4.35 -> 96.74, while Non-Meta improves by only +4.58). However, this shortcut gain is still counted toward the Table 1 averages and the overall "+5.69" conclusion. I recommend reporting the Non-Meta score (or both All and Non-Meta) in the headline results; otherwise the method's true reasoning improvement is likely overstated.

Response: We agree that reporting Non-Meta scores better reflects mathematical reasoning improvements. Following your recommendation, we have revised Table 1 in our manuscript to report LiveMath Non-Meta scores for all methods and recomputed the averages. The overall "+5.69" conclusion is then updated to "+5.32", and we also revised all relevant numerical statements accordingly. The revised Table 1 is as follows:

ModelSkillSpreadsheetSearchQALiveMath (Non-Meta)ALFWorldOfficeQADocVQAAvg
Qwen3.8-Flash-NextNo skill48.5867.5047.5081.5323.4890.3759.83
Init skill45.5567.9747.0882.4730.7591.3860.87
Trace2Skill69.6669.9346.6785.0749.8391.3168.75
SkillOpt-Lite73.7572.4347.9286.1960.3091.7172.05
SkillOpt72.5172.5747.0884.7059.8091.2471.32
SkillCome75.7174.1448.7591.7963.6892.7874.48
DeepSeek-V4-Pro-0813No skill47.7866.5746.2566.9853.71-56.26
Init skill48.4067.0745.0079.1054.56-58.83
Trace2Skill54.9869.2147.0880.0446.45-59.55
SkillOpt-Lite59.0869.2948.3380.6054.73-62.41
SkillOpt57.3870.8637.9284.3256.76-61.45
SkillCome62.1072.5049.1689.1859.29-66.45
GLM-5.3No skill45.7367.9352.5086.9420.78-54.78
Init skill52.4468.5755.4291.2327.84-59.10
Trace2Skill43.1570.5055.8388.8127.03-57.06
SkillOpt-Lite68.2377.4341.6788.4346.62-64.48
SkillOpt66.9074.2951.6792.3546.28-66.30
SkillCome71.5377.0765.0095.9048.65-71.63
DeepSeek-V4-FlashNo skill47.3364.8552.5059.515.41-45.92
Init skill48.5865.4354.5856.9111.66-47.43
Trace2Skill51.0771.5050.8379.1012.84-53.07
SkillOpt-Lite57.3868.0054.5861.0124.66-53.13
SkillOpt59.1671.5747.0874.4426.01-55.65
SkillCome62.5473.3657.0884.1450.51-65.53
Qwen3.6-35B-A3BNo skill41.9966.0040.0066.0437.1689.7756.83
Init skill40.7566.2348.3379.4838.6890.1760.61
Trace2Skill44.9369.2148.7575.5635.6489.1060.53
SkillOpt-Lite56.9367.5748.3381.3437.8490.1163.69
SkillOpt57.6567.3651.6783.2134.6389.7764.05
SkillCome61.4871.2953.3386.3840.0391.2467.29

As shown, removing the meta-option questions reduces the gains of skill evolution methods on LiveMath. Nevertheless, SkillCome still achieves the best LiveMath performance across all five models, confirming that our improvements are not solely from the shortcut.

Question 1: After excluding the Meta subset from LiveMath (or fixing this labeling artifact), what happens to the average advantage in Table 1 and the +5.69 gain over SkillOpt? Would the authors be willing to report both All and Non-Meta results in the main text?

Response: As discussed above, excluding the Meta subset reduces the gains of skill evolution methods. And the average gain over SkillOpt changes from +5.69 to +5.32 points. We have updated Table 1 in the main text to report Non-Meta scores. We also expanded Table 4 in Appendix C.1 to include the complete All / Meta / Non-Meta results for all methods:

ModelSkillAllMetaNon-Meta
GLM-5.3No skill36.0814.6752.50
Init skill38.4416.3055.42
Trace2Skill36.7911.9655.83
SkillOpt-Lite65.0995.6541.67
SkillOpt45.0536.4151.67
SkillCome67.9271.7465.00
DeepSeek-V4-FlashNo skill31.604.3552.50
Init skill35.3810.3354.58
Trace2Skill33.2510.3350.83
SkillOpt-Lite33.736.5254.58
SkillOpt68.6396.7447.08
SkillCome74.2996.7457.08
Qwen3.8-Flash-NextNo skill33.7315.7647.50
Init skill35.1419.5747.08
Trace2Skill31.6011.9446.67
SkillOpt-Lite41.7533.7047.92
SkillOpt49.5352.7247.08
SkillCome63.6883.1548.75
DeepSeek-V4-Pro-0813No skill28.074.3546.25
Init skill31.3713.5945.00
Trace2Skill33.2515.2247.08
SkillOpt-Lite32.0810.8848.33
SkillOpt48.3561.9637.92
SkillCome53.3058.7049.16
Qwen3.6-35B-A3BNo skill26.659.2440.00
Init skill30.196.5248.33
Trace2Skill31.609.2448.75
SkillOpt-Lite29.013.8048.33
SkillOpt69.8193.4851.67
SkillCome71.4695.1153.33
AuthorsOpenReview ↗

Response to Reviewer wZgb (Part2)

Question 2: The validation sets are generally very small (LiveMath has only 35 questions). Is the greedy acceptance criterion prone to overfitting validation noise? Is the validation step at which the best skill is selected stable across runs?

Response: We acknowledge that small validation sets of certain datasets like LiveMath can indeed introduce noise. However, the greedy acceptance criterion remains robust and is not prone to overfitting. Because the changes in validation scores brought by skill edits are substantially larger than the fluctuations of a small validation set. Effective skill revisions typically yield large gains over 10%, which easily surpasses the validation noise.

For the second question, we conducted 4 independent optimization runs on LiveMath. The best skills are selected at steps 8, 8, 8, and 9, respectively. This shows that the selection step is generally stable.

We sincerely thank the reviewer for this insightful question. In practical engineering, increasing the validation set size or validating multiple times per step should be effective strategies to enhance the stability of skill evolution. We have included this discussion in the Experimental Setting section of our revised manuscript.

AuthorsOpenReview ↗

Rebuttal Summary

Dear Area Chair and Reviewers,

We appreciate the time and effort all reviewers dedicated to evaluating our work. We are encouraged that all reviewers unanimously recognized the following strengths of SkillCome:

  • Clear motivation and effective method design. Reviewer b2iP highlighted that our motivation is "clear and intuitive", and contrasting successful and failed attempts provides a "more informative signal". Reviewer yMx8 also recognized the informative optimization signals and further described combining these signals with historical evidence as a "sensible design". And Reviewer wZgb highlighted that our method is well-motivated and the two components shows "clear synergistic effect".
  • Comprehensive evaluation and practical value. All three reviewers acknowledged our evaluation across five models and six benchmarks as broad, extensive, and comprehensive. They also recognized our additional analysis on transferability, initialization, and agent harnesses.
  • Efficiency and reproducibility. Reviewers yMx8 and wZgb highlighted our lower optimizer token usage. Reviewer wZgb also recognized that our "Reproducibility materials (code, prompts, data splits, hyperparameters) are complete".

In response to the reviewers' suggestions, we have conducted additional experiments and revised the paper (changes highlighted in blue). A summary of the key updates follows:

  1. Additional ablations and ablation on stronger model. We conducted new ablations of "SkillCome w/o Contrast Reflection" and "SkillCome w/ Combined Memory" to further prove the effectiveness of the two components of SkillCome (yMx8). We also included the following ablations: removing the contrast memory and the failure memory separately, a variant with only contrast reflection, the ranking strategy that prioritizes contrast edits, and the effect of the group size (b2iP). In addition, we further expanded these experiments on the stronger model Qwen3.8-Flash-Next (b2iP).

  2. Clearer comparisons and discussion. We clarified how the headline improvements are averaged across models (b2iP). We now report LiveMath Non-Meta scores in the main Table 1, with complete All / Meta / Non-Meta results for every method in Appendix C.1 (yMx8, wZgb). Finally, we added a discussion of ReasoningBank in the Related Work section, and clarified its differences with our method (b2iP).

  3. More implementation and training details. We specified the rollout allocation of each method, validation evaluations, memory capacities, skill-document lengths, and stability of SkillCome across optimization runs (yMx8). Details on how validation set size affects skill selection are also provided in the Implementation Details section (wZgb).

We believe these revisions significantly strengthen the paper and address the reviewers' concerns. We have responded to each reviewer individually below. Unfortunately, we have not yet received follow-up feedback from any of the three reviewers. Respectfully, we ask the reviewers and Area Chair to consider this additional evidence in your final assessment, and we remain available to address any remaining concerns.

Best regards,

Authors of #16200

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.