acceptodds
Under review as a conference paper at ICLR 2027

Evolving Agent Expertise from Conversational Feedback

Abstract

Deployed Large Language Model (LLM) agents accumulate extensive interaction logs in which user feedback implicitly signals where and how the model should improve. Learning from such feedback is difficult because the signal is both weak and mixed. It is weak because it arrives as free-form text with no success oracle, and because each remark covers only a fragment of what a task requires, so a reusable lesson emerges only after many remarks are aggregated. It is mixed because a single remark may point to a flawed procedure, a missing fact, or a skill applied where it does not belong. Existing methods each maintain a single kind of store and therefore absorb only part of this signal. Memory systems retain facts but never distill them into procedures, while skill-evolution methods distill procedures but discard the facts and never learn when a skill should not be applied. To address this, we present SkillWeave, which learns from feedback logs alone, without a success oracle or parameter updates, by jointly evolving a skill library, an evidence-grounded knowledge base, and the applicability conditions of each skill. Logs are processed in batches. Each batch first extracts missing facts, then groups logs into task topics, and finally creates or revises one skill per topic together with the conditions that govern its use. On user-feedback benchmarks covering legal, academic, and open-domain tasks, SkillWeave outperforms eight memory and skill-evolution baselines across three backbone LLMs, and it stays competitive on long-document tasks.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
LLM Agent, Skill, Self-Improvement

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_VaGZOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

SkillWeave studies how a frozen LLM agent can improve from accumulated conversational feedback without receiving reference answers or task-success labels during adaptation. It jointly maintains procedural skills and a knowledge base. Feedback logs are grouped into topics to create or revise skills, while factual gaps are addressed through evidence-grounded knowledge accumulation. Skills also include applicability conditions and optional knowledge-retrieval instructions. The main contribution is a concrete design for coordinating procedural and factual learning from mixed, partial feedback. The paper reports the highest MemoryBench average among the evaluated methods on each of three backbones and competitive results on ExpertLongBench.

Critical strengths
  1. The paper offers useful, concrete methodological contributions. Distinguishing factual corrections, procedural lessons, and applicability feedback is a sensible response to conversational feedback that mixes these signals. Aggregating related feedback before revising a skill, retaining supporting knowledge, and recording when a skill should not apply form a coherent implementation. I see these choices as meaningful incremental contributions with practical value.

  2. The empirical evaluation provides encouraging evidence of usefulness. The MemoryBench results cover several domains and three backbones, with additional evaluation on long-document tasks. The ablations and repeated accumulation runs help establish that the complete system is useful and that its reported gains are not based only on a single accumulation run. This makes the method worth further consideration, even though the experiments below are needed to identify what drives its advantage.

Critical weaknesses
  1. The contribution needs clearer positioning within skill–memory co-evolution. At a conceptual level, SkillWeave coordinates the evolution and use of procedural skills and factual memory. The broad contrast that existing approaches maintain only one kind of store does not adequately place it within related approaches. For example, MSCE, Sections 3–4 develops memory and skills together, including environmental knowledge that informs skill use. Mem²Evolve, Section 3 couples experience memory with reusable tools and expert agents. These methods have different representations and feedback protocols, so they should not be treated as equivalent to SkillWeave. However, the paper needs to identify which new mechanism or capability follows from its conversational-feedback setting, beyond maintaining complementary stores. This affects how much novelty can be assigned to the framework itself, while leaving room for its specific designs to be valuable. (Lines 047–060 and 079–101.)

  2. The current controls do not sufficiently isolate the value of the proposed coordination and topic aggregation. The knowledge-base ablation and raw-Wikipedia controls are useful, but they do not establish whether an existing skill learner given the same curated knowledge could obtain similar gains. Separately, removing clustering changes both the update procedure and the resulting skill library: the paper reports 36 skills versus 8 for the full method. The resulting performance gap therefore cannot be attributed specifically to grouping feedback by topic. These are fixable evidence gaps, but they concern the central explanation for why this method improves over simpler alternatives. (Table 2; lines 351–374.)

Questions for authors
  1. What specifically distinguishes SkillWeave's skill–memory co-evolution from related approaches such as MSCE and Mem²Evolve? Please compare the stored representations, available feedback, use of success signals, knowledge sources, and dependencies between skill and memory updates. In particular, explain which parts of the proposed coordination are new and which adapt existing ideas to conversational feedback. A distinction based only on natural-language feedback would need care: MSCE can convert textual user feedback into a scalar update signal. This comparison should clarify the contribution rather than assume that all related methods require reference answers. A direct benchmark comparison would be informative where protocols can be aligned, but the requested conceptual distinction does not require transplanting incompatible original settings. (Lines 047–060, 089–110; MSCE, Section 3 and Appendix B.3; Mem²Evolve, Section 3.)

  2. Does SkillWeave retain an advantage over a simple combination of AutoSkill and the same knowledge base? Please compare AutoSkill, AutoSkill plus knowledge retrieval, and SkillWeave on MemoryBench. For the latter two, share a frozen knowledge base constructed during training using the same permitted knowledge sources, without held-out queries or their reference answers; let the augmented AutoSkill retrieve relevant entries alongside its own skills. Use the same backbone, training logs, and held-out queries, and align retrieval access, tool interfaces, length control, and total context budget as closely as feasible. Report task scores and uncertainty, actual tokens and calls, and construction and inference costs, accounting explicitly for the shared knowledge-base cost. This would isolate the benefit of skill organization, updating, and use after controlling factual information; it would not test the knowledge-base construction method itself. (Lines 302–320; Table 2 and lines 356–374.)

  3. Does grouping feedback by topic improve held-out performance beyond the effects of batching and skill capacity? Please compare topic-based grouping with random grouping while retaining the same training logs, feedback text, and knowledge base. Match group sizes, update opportunities, skill capacity, and skill-generation budget as closely as feasible, and report any remaining differences. Repeat the random grouping across seeds and report the performance difference and uncertainty. This directly tests whether semantic grouping helps combine partial feedback into reusable procedures, which the current 36-versus-8-skill comparison does not isolate. (Lines 039–045 and 232–242; Table 2 and lines 351–356.)

6 comments
AuthorsOpenReview ↗

Response to Reviewer VaGZ (Part 1/2: W1, Q1)

We sincerely thank you for recognizing the practical value of separating components, and for proposing clear positioning dimensions and rigorous controlled experiments. We address W1/Q1 here and W2/Q2 and Q3 in Part 2/2.

1. Positioning Within Skill and Memory Co-Evolution: MSCE and Mem²Evolve (W1 & Q1)

Thank you for highlighting these concurrent preprints. We have refined Section 1, Section 2.2, and Appendix I (Table 9) to position SkillWeave explicitly alongside MSCE and Mem²Evolve across the five dimensions you raised:

DimensionMSCEMem²EvolveSkillWeave (Ours)
RepresentationsL1/L2/L3 hierarchy + promoted skillsExpert agents, code tools + experiencesTask procedures (Avoid when) + factual KB
FeedbackEnv. verifier reward (+ optional coach diagnosis on failed traces) & step reflectionExecution trajectories only (no user text)Free-form user follow-up remarks only
Success SignalScalarized return Ri∈[−1,1]R_i\in[-1,1] & gain gateLLM-judged 0/1 trajectory success gateNone (content-routed, no scalar/gate)
Knowledge SourceInternal agent trajectories onlyParametric + web search for tool codeSession logs + verified Wikipedia evidence
Update CouplingL1 to L2 to skills & L3 env. priorsExperience guides new tool/agent creationPer-batch fact check + topic skill rewrite

Maintaining complementary stores, aggregating across multiple episodes before inducing procedures (such as MSCE's nmin⁡n_{\min} episodes or SkillOpt's minibatches), and tracking applicability boundaries are shared principles across recent self-evolving agent work, which we explicitly credit in Section 2.2 and Appendix I. Notice also the difference in feedback protocol and setting: MSCE's main experiments on multi-step tool environments (EvoAgentBench) rely on environment verifier rewards, and its simulated textual feedback study (their Appendix 10.2) invokes a coach model on verifier-flagged failed trajectories with access to full internal traces, verifier results, and stored reflections before quantifying the diagnosis into a scalar return RiR_i (because both MSCE and Mem²Evolve operate over online multi-step action-observation trajectories rather than static dialogue logs, and Mem²Evolve has no public code, their native protocols do not directly transplant to offline conversational logs). More fundamentally, converting textual feedback into a scalar RiR_i effectively governs whether to promote or retire a policy, but scalar compression discards which specific domain fact was missing and how a procedure should be rewritten. For offline user logs with no verifier or re-execution rollouts, SkillWeave introduces (1) evidence-grounded gap distillation against an external reference corpus to prevent unverified claims or hallucinations from entering long-term memory, and (2) soft Avoid when self-judgment in a progressive-disclosure catalog excluded from BM25 scoring, preventing the severe false drops caused by lexical hard filtering.

AuthorsOpenReview ↗

Response to Reviewer VaGZ (Part 2/2: W2, Q2, Q3)

2. Controlled Comparison Against AutoSkill Sharing the Same Frozen KB (W2 & Q2)

Thank you for designing this clean protocol to isolate skill learning after controlling factual information. Following your protocol, we compared AutoSkill (AS), AutoSkill + Frozen KB (AS+KB), and SkillWeave (SW) across all three domains and their mean (MB Avg.) on DS-V4-Flash using identical training logs and held-out queries. AS+KB shares the exact frozen KB built during training (with zero access to held-out queries or references) and retrieves the top-5 entries via BM25 alongside its own skills (capped at 12,500 chars, averaging 2,298 / 1,492 / 1,701 injected tokens per query on Legal / Academic / Open-Domain), with the shared KB build cost (1.18 / 0.98 / 1.30) explicitly included in AS+KB:

DomainAutoSkillAutoSkill + Frozen KBSkillWeaveSW − (AS+KB) [95% CI]Build Calls / Cost (AS+KB vs. SW)Test / Total Cost (AS+KB vs. SW)
Legal41.546.150.1+3.9 [+1.0, +6.9] (p=.009p=.009)2,640+KB (5.49) vs. 996 (1.60)0.66 / 6.15 vs. 0.25 / 1.85
Academic67.167.668.8+1.2 [-0.8, +3.2] (p=.23p=.23)3,320+KB (5.78) vs. 1,357 (1.33)0.29 / 6.07 vs. 0.40 / 1.73
Open-Domain63.063.374.6+11.3 [+7.6, +15.1] (p<10−8p<10^{-8})11,816+KB (6.96) vs. 1,591 (1.69)0.36 / 7.32 vs. 0.51 / 2.21
MB Avg.57.259.064.5+5.5

Augmenting AutoSkill with the exact same curated KB improves Legal (though still trailing KB-free SkillWeave w/o KB at 47.1), yet SkillWeave retains a substantial lead across domains (+5.5 average) at under one-third of the total cost. This isolates the value of inducing procedures from full query, response, and feedback triples (whereas AutoSkill extracts from user queries alone) and invoking KB search on demand via skill directives rather than unconditionally prepending retrieved passages onto non-factual writing tasks.

3. Topic Grouping vs. Capacity- and Budget-Matched Random Grouping (W2 & Q3)

Thank you for noting that comparing 36 versus 8 skills conflates topic grouping with batching and skill capacity. On DS-V4-Flash Legal, holding training logs, feedback text, and the frozen KB identical, we ran 3-seed random-grouping controls: within each batch (b=50b=50), clustering and topic-extraction calls run as normal to match call budgets, after which newly assigned cases have their group labels randomly permuted across groups and receive neutral identifiers (Group 1, Group 2, ...) so mismatched titles do not mislead generation. This matches per-batch group-size distributions, nmin⁡=5n_{\min}=5 update opportunities, and skill capacity. We also ran a 3-seed control keeping semantic grouping with neutral identifiers:

Setting (DS-V4-Flash Legal)Score#SkillsΔ\Delta [95% CI]
SkillWeave (semantic grouping + explicit topic names)50.18ref.
Random grouping + neutral IDs (3 seeds)48.1 ± 2.77 to 9−2.0 [-5.8, +1.9]
w/o clustering (log-by-log updates)47.836−2.2
Semantic grouping + neutral IDs (3 seeds)45.8 ± 0.53 to 8−4.3 [-7.3, -1.3] (p=.004p=.004)
Single topic (one coarse topic per cluster)45.08−5.1

(Per-batch permutation matches within-batch group sizes, while across 20 batches cross-batch topic merging yields a minor natural difference of 7 to 9 vs. 8 final skills.)

Together with Table 2, these controls clarify how grouping works. While batching into compact libraries avoids the fragmentation of 36 single-log skills, random grouping lowers mean score and introduces severe seed-to-seed instability (±2.7, far exceeding SkillWeave's retrain spread in Appendix F), showing that aggregating semantically related remarks is essential for stable induction. Moreover, the significant drop under Semantic grouping + neutral IDs demonstrates that extracting explicit task-type names and descriptions provides indispensable semantic anchors for defining genre boundaries and BM25 catalog retrieval, consistent with the sharp drop of Single topic when distinct sub-genres are merged under one broad label. We have added both controlled studies to Appendix L.3 (Table 17) and Appendix L.4 (Table 18).

Thank you again for these constructive experimental designs, which substantially sharpened both our positioning and our empirical attribution.

Thank you for the additional experiments and the work you have put into addressing my comments. The shared-KB comparison and the clearer positioning have addressed much of my concern. I would like to see one further control: random grouping with explicit topic names generated from the randomized groups, using the same naming and retrieval procedure as full SkillWeave, with comparable skill capacity and update budgets. Please report the score difference and its uncertainty over the same three seeds. This would help distinguish the effect of semantic grouping from that of naming. Once this comparison is included, I would be willing to raise my rating to 3.

Reviewer_ebaROpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

SkillWeave learns external agent state from conversational feedback while keeping the backbone frozen. It grounds factual entries in retrieved Wikipedia evidence, groups interaction logs into clusters and topics, and creates or revises skills containing procedures, applicability exclusions, and knowledge-retrieval instructions. At inference, the agent selects skills from a retrieved catalog, optionally queries the knowledge store, and applies a response-length calibration step. Evaluation uses simulated user feedback on MemoryBench and ExpertLongBench, with three backbones, component ablations, online accumulation, cross-backbone skill transfer, cost measurements, and five repeated full builds on one backbone.

Critical strengths
  1. The empirical advantage is consistent and supported beyond a single run. SkillWeave achieves MemoryBench averages of 64.5, 63.9, and 69.0, compared with 60.0, 60.5, and 65.1 for SimpleMem, the strongest reported baseline on each backbone. Appendix F repeats the full accumulation/inference pipeline five times with fixed training logs: the SkillWeave–SimpleMem difference is +3.9 points with a reported interval of [2.1, 5.8]. The cross-backbone transfer and online accumulation results further support the practical value of the learned artifacts. The more modest ExpertLongBench results are reported without claiming uniform superiority.

  2. Several controls address plausible alternative explanations. Table 2 includes removing the knowledge base, retrieving Wikipedia directly, giving raw logs plus Wikipedia to BM25, removing clustering, and hiding negative triggers. Appendix D also tests the agent system prompt alone. These results support a benefit from organizing feedback rather than merely adding the same raw information or changing the system prompt. The method also offers a concrete way to use natural-language feedback without task scores being exposed during adaptation.

Critical weaknesses
  1. The component evidence does not fully separate co-evolution from other effective pipeline changes. On Legal, removing response compression costs 3.2 points, exceeding the 2.3-point negative-trigger ablation. Displaying exclusions helps, but their evolution is not isolated from a fixed applicability description, and baseline use of the same calibration is not reported. Most component tests concern this domain/backbone. In Open-Domain, transcripts are inserted directly into the KB, with entries up to 12,000 characters (Section 4.2, Table 4); per-subtask scores and per-method retrieved token counts would clarify the contribution of transcript retrieval. This is not evidence that baselines lacked the corpus. The full-system gain is credible, while its attribution specifically to evolving applicability and coordinated procedural/factual learning remains incomplete. Because coordinated evolution is a central contribution claim, this unresolved attribution is consequential to the acceptance decision, even though the complete system improves performance.

  2. Some claims about competing methods are inaccurate, weakening the novelty and efficiency framing. The introduction's claim that existing methods maintain only one store is too broad: MemSkill explicitly has separate memory and skill banks, with applicability information in its skills. SkillWeave's more specific distinction is factual grounding plus task-execution skills learned without reward-based adaptation. Likewise, Section 6.3 attributes competitors' high cost to log-by-log updates, whereas the original SkillOpt uses rollout batches and minibatch reflection. The adapted comparison may still be useful, but its update schedule and cost should not be presented as intrinsic to the original method. MemSkill, Sections 3.1–3.3; SkillOpt, Method.

Questions for authors
  1. Is response-length calibration shared with any baselines, and do existing results distinguish updated negative triggers from fixed applicability descriptions? For Open-Domain, could you report the sub-benchmark scores and actual retrieved-context sizes for SkillWeave and the strongest memory baselines? These clarifications would separate the contribution of learned skills/applicability from compression and transcript retrieval.
  2. What exact SkillOpt rollout/reflection batch sizes, revision schedule, and validation schedule produced the reported 19k calls? Please distinguish departures required by the feedback-only setting from choices made for this implementation. This could materially affect the claimed efficiency advantage over skill-evolution methods.
3 comments
AuthorsOpenReview ↗

Response to Reviewer ebaR (Part 1/2: W1, Q1)

We sincerely thank you for the thoughtful suggestions. We address W1/Q1 here and W2/Q2 in Part 2/2.

1. Component Attribution and Open-Domain Context Analysis (W1 & Q1)

Thank you for these thoughtful questions on isolating each component's contribution. Regarding response-length calibration, we first clarify that baselines in Table 1 run in their original configurations without post-hoc compaction. To check whether sharing this step benefits baselines, we applied the exact same domain-level length compaction (identical prompt, explicit-length guards, and training p75/p90 thresholds) to Base, SimpleMem, A-Mem, and AutoSkill on DS-V4-Flash Legal, yielding 38.8, 42.7, 45.4, and 41.2 (vs. original 39.5, 42.7, 45.5, and 41.5), all slightly negative to zero. Meanwhile, SkillWeave without length statistics (46.9) still beats every baseline, and under the cross-provider judge gpt-6-luna both default SkillWeave (41.9) and its uncompacted variant (41.1) rank first by a wide margin, confirming that system gains do not stem from post-hoc compression.

To isolate updated negative triggers from fixed applicability descriptions, we added retraining controls on DS-V4-Flash Legal where negative triggers are frozen at their initial first-batch text (while methodology bodies continue to be rewritten across batches) or omitted during training, alongside a cross-backbone hard-filter ablation on Qwen3.5-27B Legal:

Backbone & DomainVariantScoreΔ\Delta
DS-V4-Flash LegalSkillWeave (co-evolving exclusions)50.1ref.
Frozen initial exclusions48.0−2.1
w/o neg. triggers (exclusions hidden at inference)47.8−2.3
Trained without exclusions47.1−3.0
Hard neg. filter (lexical-overlap drop)43.1−6.9
Qwen3.5-27B LegalSkillWeave (co-evolving exclusions)52.7ref.
Hard neg. filter (lexical-overlap drop)40.8−11.9

Displaying soft exclusions in the catalog prevents cross-genre misapplication, whereas hard lexical filtering discards applicable skills and causes severe degradation across backbones. Crucially, freezing exclusions at their initial small-batch draft recovers little over hiding them altogether, confirming that iteratively refining exclusions as broader task evidence arrives is essential (Appendix H illustrates how all four exclusions are progressively replaced with sharper boundaries across chunks). We have added these controls to Appendix L.2 (Table 16).

For Open-Domain, all 8 sub-benchmark scores appear in Appendix K (Table 14). Below we report scores on the three DialSim subsets (BigBang / Friends / TheOffice, 17 items each), Locomo (50 items), the remaining 4 non-memory tasks (233 items), and the Open-Domain (OD) total (334 items), alongside median injected tokens per query on the 4 dialogue-memory tasks (Memory-4 Tok) versus the 4 non-memory tasks (Other-4 Tok) on DS-V4-Flash:

MethodDialSim BB / Fr / Of (17 ea.)Locomo (50)Other 4 Tasks (233)OD Total (334)Memory-4 TokOther-4 Tok
RAG-BM2511.8 / 0.0 / 17.645.865.153.83,9104,019
LightMem0.0 / 0.0 / 0.044.076.059.6148213
A-Mem11.8 / 5.9 / 5.932.675.458.74884,048
AutoSkill11.8 / 0.0 / 5.942.180.063.05,98526
SimpleMem23.5 / 0.0 / 23.556.082.868.5200191
SkillWeave82.4 / 29.4 / 47.150.984.474.63,392 (KB)1,596 (Skill)

Larger retrieved context alone does not drive performance: RAG-BM25 and AutoSkill inject more corpus tokens on the four dialogue-memory tasks yet trail far behind, and on the remaining four non-memory tasks (233 items, zero transcript retrieval) SkillWeave still achieves the highest score purely via learned skills. Furthermore, when we cap cumulative KB retrieval per query to SimpleMem's exact median budget (919 characters, ~253 median tokens), SkillWeave scores 41.2 / 23.5 / 23.5 on DialSim and 42.4 on Locomo, still outperforming SimpleMem on both the four memory tasks (35.9 vs. 35.7) and Open-Domain overall (72.1 vs. 68.5). Likewise, when AutoSkill shares our exact frozen KB, SkillWeave still leads by +11.3 points (74.6 vs. 63.3, p<10−8p<10^{-8}). We have reported these statistics and budget-matched controls in Appendix M.1 (Table 19) and Appendix L.3 (Table 17).

AuthorsOpenReview ↗

Response to Reviewer ebaR (Part 2/2: W2, Q2)

2. Positioning Relative to MemSkill and SkillOpt & Breakdown of the 19k Calls (W2 & Q2)

Thank you for pointing out where our phrasing can be made more precise. Our "single-store" shorthand in Section 1 referred to methods that do not jointly maintain downstream task-execution skills and an evidence-grounded domain knowledge base, and we have sharpened Sections 1 and 2.2 to state the exact distinction with MemSkill: MemSkill relies on downstream task rewards (an RL controller plus a ground-truth hard-case buffer) to learn memory-operation skills governing how episodic memories are written and consolidated, whereas SkillWeave learns task-execution procedures directly from reward-free conversational feedback and verifies missing domain facts against external Wikipedia evidence.

For SkillOpt, we appreciate the opportunity to clarify our exact experimental setup. In Section 6.3, "log by log" described AutoSkill, whereas our SkillOpt baseline runs the original paper's default batched schedule unchanged (4 epochs, rollout batch size 40, reflection minibatch size 8, and a 2:1 split of the 601 feedback-bearing Legal logs into 401 training and 200 validation items across 44 steps). Because no ground-truth scorer exists, evaluating one rollout requires two LLM calls (one target-model answer plus one judge call against the recorded user feedback), yielding the exact 19,437 build calls:

Build PhaseExact CalculationCalls (Share)
Selection-gate validationInitial skill (1) + 38 candidate skills (5 steps hit cache, 1 yielded no edits) × 200 items × 215,600 (80.3%)
Training batch rollouts4 epochs × 401 items × 23,208 (16.5%)
Slow-update rollouts & reflection / merge / rankSlow-update sampling (240) + minibatch reflection, merging, and ranking (389)629 (3.2%)

Regarding required adaptations versus implementation choices, scoring rollouts with an LLM judge against recorded user feedback and holding out a validation split from training logs are required by the oracle-free feedback setting. By contrast, setting the optimizer and judge to the tested backbone itself (matching SkillOpt's target-matched optimizer setting in their Table 5) is a fairness choice so that every method uses only the backbone during accumulation. Thus, SkillOpt's build cost is driven by candidate-by-candidate full-validation gating (80.3% of calls) rather than update granularity. Even excluding all 15,600 validation calls, its remaining rollout and reflection calls (3,837) are nearly 4× SkillWeave's entire build phase (996 calls). We have clarified Section 6.3 so that AutoSkill and SkillOpt are distinguished explicitly and included this breakdown in Appendix M.2 (Table 20).

Thank you again for your careful reading and constructive suggestions, which helped us sharpen both our related-work discussion and our attribution analysis.

Dear Reviewer ebaR,

As the discussion period approaches its end, we wanted to kindly follow up to check whether our responses and manuscript revisions have addressed your questions. Following your feedback, we have added baseline length-compaction controls, frozen negative-trigger retraining and cross-backbone hard-filter ablations, and Open-Domain context-budget matching controls (Appendices K, L, and M.1), as well as sharpened our positioning of MemSkill and SkillOpt with an exact breakdown of the 19,437 build calls (Sections 1, 2.2, 6.3, and Appendix M.2).

If these additions have resolved your concerns, we would be very grateful if you would consider raising your score. We remain happy to discuss any further questions, and thank you again for your careful review!

Reviewer_KymsOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces SkillWeave, a framework that evolves agent expertise from conversational feedback in deployed interaction logs, with no success oracle and no parameter updates. It targets a signal that is both weak and mixed: feedback is free-form and unlabeled, each remark covers only a fragment of a task, and it may indicate a flawed procedure, a missing fact, or a skill applied where it does not belong. Prior methods keep a single store — memory systems retain facts but never distill procedures, while skill-evolution methods distill procedures but discard facts and never learn when a skill should not apply — so each absorbs only part of this signal. SkillWeave jointly evolves a skill library (methodology, negative triggers for when not to apply it, length percentiles) and an evidence-grounded knowledge base: logs arrive in batches and pass through fact extraction via Wikipedia retrieval and reflection, incremental clustering, adaptive topic extraction, and skill rewriting from all cases in a topic. At inference the top-5 catalog entries with "Avoid when" clauses are retrieved by BM25, the model self-judges applicability before expanding a skill, and over-long outputs are compressed to a learned length budget. SkillWeave introduces a framework absorbing procedural, factual, and applicability lessons into two co-evolving stores; a batch pipeline that aggregates sparse feedback before writing skills, cutting build cost; and state-of-the-art results on MemoryBench and ExpertLongBench across three backbones.

Critical strengths
  • The motivation is concrete rather than generic, and targets a real problem. Deployed agents accumulate interaction logs in which user feedback is a weak and mixed learning signal: free-form and unlabeled, covering only a fragment of any single task, and pointing variously to a flawed procedure, a missing fact, or a skill applied where it does not belong. This three-way decomposition is genuinely useful — it directly motivates the paper's two-store design and the inclusion of negative triggers, rather than serving as post-hoc framing.
  • Reproducibility is above the norm for this line of work. Code is released through an anonymous link, hyperparameters are listed in full (Table 4), prompt templates are provided in the appendix, and both API unit prices and hardware are reported so that the cost analysis can be independently checked.
  • The authors report more than single-run numbers. Full training is repeated five times, with mean and standard deviation plus 95% confidence intervals reported, and the study additionally includes cross-backbone skill transfer and a pseudo-feedback variant in which the simulator does not see the reference material.
Critical weaknesses
  • All evaluation is model-based, with no human validation. Every score comes from an LLM judge, including open-ended sub-benchmarks graded by rubric; there is no human A/B comparison and no judge–human agreement measure, so it remains unverified whether the reported ranking tracks actual response quality. The risk is compounded by family overlap: the judge (DeepSeek-V4-Pro) belongs to the same model family as a tested backbone (DeepSeek-V4-Flash), and the user simulator (Qwen3-32B) belongs to the same family as another (Qwen3.5-27B), where self-family preference is well documented. Notably, the largest single gain in the paper (+12.9 on ExpertLongBench) falls on the backbone sharing a family with the simulator. A re-run of the key columns with a judge from a different provider, or human evaluation on a sampled subset, would settle this.

  • Only domain-level aggregates are reported, and the evaluation is limited to single-turn generation. MemoryBench contains 19 sub-benchmarks whose metrics range from F1 and accuracy to ROUGE-L, METEOR, and LLM-judge scores, yet the paper reports only three domain scores. It is therefore impossible to tell whether the gains are broad or concentrated in a few sub-benchmarks — a concern that matters because the largest gain is on Open-Domain, where four of eight sub-benchmarks are conversation-memory tasks. Separately, every evaluated task is single-turn retrieval-augmented generation: there are no multi-step, tool-using agent scenarios and no reasoning-heavy benchmarks (math, code, multi-hop planning). Whether a skill library helps when the bottleneck is reasoning rather than convention recall is thus untested.

  • The single largest contributing component is an inference-time length-calibration trick, confounded with the judge's length preference. On Legal, disabling length statistics costs 3.2 points — larger than removing the knowledge base (2.9), the negative triggers (2.3), or clustering (2.2), i.e. larger than any component the paper presents as a conceptual contribution. Since many sub-benchmarks are graded by an LLM judge that is known to penalize verbose output, it is unclear how much of this gain comes from correcting genuine output drift and how much from accommodating a judge bias. Controls that would separate length control from the self-refinement effect of an extra revision call — always compress, compress to a random threshold, compress to a fixed length — are missing.

Questions for authors

On the judge:​ Could the author re-run the main columns with a judge from a different provider, or report judge–human agreement on a sampled subset? The current judge (DeepSeek-V4-Pro) and user simulator (Qwen3-32B) share model families with two of the three tested backbones, and no human evaluation is reported anywhere.

On the evaluation coverage:​ Could the author provide per-sub-benchmark scores for all 19 MemoryBench sub-benchmarks, so that it is possible to see whether the gains are broad or concentrated in a few tasks? In addition, would the method still help on at least one multi-step tool-use or reasoning-heavy setting, rather than single-turn retrieval-augmented generation?

On length calibration:​ Could the author add controls that compress always, to a random threshold, or to a fixed length, in order to separate genuine length control from the self-refinement effect of an extra revision call? How much of the 3.2-point gain on Legal survives under a judge that is not sensitive to output length?

3 comments
AuthorsOpenReview ↗

Response to Reviewer Kyms (Part 1/2: W1, W2, Q1, Q2)

We sincerely thank you for recognizing our three-way decomposition of conversational feedback, reproducibility, and multi-run and cross-backbone transfer experiments. We address W1/Q1 and W2/Q2 here, and W3/Q3 in Part 2/2.

1. Deterministic Metric Coverage and Cross-Provider Judge Re-Evaluation (W1 & Q1)

Thank you for raising this important concern regarding evaluation reliability. First, we would like to respectfully clarify that our evaluation does not rely solely on an LLM judge. As listed in Appendix C.1, 8 of the 19 MemoryBench sub-benchmarks use deterministic or exact-match metrics: ROUGE-L on three LexEval sets, F1 on Locomo, METEOR on WritingPrompts, and exact-match-first scoring on three DialSim sets. Even on the 5 purely deterministic sub-benchmarks with zero LLM-judge involvement (250 test items), SkillWeave significantly beats the strongest baseline SimpleMem across all three backbones (+8.3, +8.1, and +6.5 normalized points, paired tt-test p<7×10−5p < 7\times 10^{-5} on each; Appendix J.1, Table 10).

Second, we would like to clarify the role of the user simulator and model-family overlap. The user simulator (Qwen3-32B) only generates training feedback, and all nine methods on each backbone read the exact same training logs and are never scored by the simulator at test time. The +12.9 gain over Base on ExpertLongBench with Qwen3.5-27B reflects weak unadapted Base behavior on that format (Section 6.1): every method using training logs achieves a similar jump on that column, with LightMem ranking first. Moreover, on Gemini-3.7-Flash, which shares a family with neither judge nor simulator, SkillWeave's margin matches the other backbones. At the same time, we agree that calling the judge "distinct from all tested backbones" overlooked its shared family with DS-V4-Flash, and have clarified this in Section 5 (with full cross-provider verification in Appendix J.2, Table 11).

To verify judge independence, we re-scored saved predictions with OpenAI's gpt-6-luna using identical prompts, including CLEAR rubric extraction and matching on ExpertLongBench (two-pass self-consistency Spearman on 107 stratified items is 0.922). Below we compare SkillWeave against the strongest baseline per column under gpt-6-luna (MB Avg. is the 3-domain MemoryBench mean and ELB is ExpertLongBench):

BackboneMethodLegalAcademicOpen-DomainMB Avg.ELB
DS-V4-FlashBest Baseline33.455.354.346.333.7
SkillWeave41.9 (+8.5)56.0 (+0.7)64.3 (+10.0)54.1 (+7.8)35.2 (+1.5)
Qwen3.5-27BBest Baseline36.554.254.548.133.9
SkillWeave44.2 (+7.7)55.2 (+1.0)62.1 (+7.6)53.9 (+5.8)33.8 (−0.1)
Gemini-3.7-FlashSimpleMem47.155.460.354.334.4
SkillWeave53.8 (+6.7)56.1 (+0.8)66.7 (+6.4)58.9 (+4.6)36.0 (+1.6)

Under this stricter cross-provider judge, SkillWeave ranks first on every MemoryBench domain across all three backbones with even larger margins, and 9-method rankings agree strongly across judges (Kendall's τ\tau on Legal / Academic / Open-Domain is 0.83 / 0.70 / 0.83 on DS-V4-Flash and 1.00 / 0.94 / 0.67 on Qwen3.5-27B, all p<0.02p<0.02), confirming our conclusions do not rely on single-provider judge bias.

2. Per-Sub-Benchmark Breakdown and Problem Paradigm (W2 & Q2)

Thank you for this helpful suggestion on evaluation coverage. We have compiled normalized scores across all 19 sub-benchmarks for every method and backbone in Appendix K (Tables 12–14). Gains span diverse task types rather than a narrow subset: beyond multi-session dialogue memory, all three backbones achieve consistent gains on judicial opinion writing, legal QA, and case summarization (which involve no dialogue history), as well as academic writing, research ideation, and science journalism editing.

Regarding math, code, and multi-step planning benchmarks, we appreciate the opportunity to clarify the fundamental difference in problem setting. From the title through Section 3, SkillWeave is designed for deployed agents learning from human users' natural-language feedback. Code, math, and sandbox tasks rely on two assumptions: exact reward signals (compilers, unit tests, or environment verifiers) and resettable environments allowing dozens of multi-step trial-and-error rollouts per task. Neither holds in real user conversations: natural-language remarks give no exact scalar reward, and real users cannot act as online annotators tolerating repeated trial-and-error attempts on the same query. SkillWeave targets expertise evolution from sparse, mixed conversational feedback without exact rewards or environment rollouts, while still using up to four autonomous tool-call rounds at inference to load skills and query knowledge.

AuthorsOpenReview ↗

Response to Reviewer Kyms (Part 2/2: W3, Q3)

3. Controlled Attribution of Length Calibration (W3 & Q3)

Thank you for raising this thoughtful question on length calibration. Before presenting the new controls, we would like to respectfully clarify two points from our existing results. First, in Table 2, the largest performance drops come from removing fine-grained topic partitioning (Single topic, −5.1) and replacing soft exclusions with hard lexical filtering (Hard neg. filter, −6.9), rather than disabling length calibration (−3.2). Even without length statistics (46.9), SkillWeave outperforms all eight baselines on Legal, meaning gains come from learned skills and knowledge. Second, we would like to clarify the direction of judge length sensitivity: prior studies on LLM judges such as Zheng et al. (2023) [1] and Dubois et al. (2025) [2] consistently document a verbosity bias favoring longer responses rather than penalizing them. Comparing SkillWeave against w/o length stats across sub-benchmarks confirms that LLM-judged subsets do not gain from shorter outputs, ruling out judge-bias accommodation and showing that calibration addresses genuine output drift on concise tasks after loading detailed skill documents.

To separate per-skill adaptive length control from the self-refinement effect of an extra revision call, we ran all three compression controls you suggested under the exact Table 2 setup on DS-V4-Flash Legal (judge: DeepSeek-V4-Pro), and also tested adding identical domain-wide fixed compaction to four representative baselines:

Control Setting (DS-V4-Flash Legal, Judge: DeepSeek-V4-Pro)ScoreΔ\DeltaTriggered Cases
SkillWeave (per-skill adaptive percentiles)50.1ref.14 / 241 (5.8%)
Always compress (unconditionally compress all responses)49.4−0.7241 / 241 (100%)
Compress to a random threshold47.8−2.3145 / 241 (60.2%)
w/o length stats (no length calibration)46.9−3.20 / 241 (0%)
Compress to a fixed length (single domain-wide percentiles)45.6−4.541 / 241 (17.0%)
Four baselines + fixed compaction (Base / SimpleMem / A-Mem / AutoSkill)38.8 / 42.7 / 45.4 / 41.2−0.7 to 0.021 to 46 / 241

The pattern across controls is clear. Applying identical fixed compaction to four baselines brings zero positive gain, ruling out self-refinement from an extra LLM revision call. Meanwhile, a single domain-wide fixed length cannot accommodate tasks spanning concise summaries (hundreds of characters) and full judicial opinions or policy essays (thousands of characters). Random thresholds severely damage long-form documents whenever a short threshold is drawn. Unconditional compression not only hurts well-proportioned answers, but also forces every query through two sequential generation passes, doubling inference calls and latency in a way that is impractical for deployment. By contrast, per-skill percentiles trigger on fewer than 6% of extreme outliers, curbing output drift with negligible overhead. Under the cross-provider judge gpt-6-luna, SkillWeave leads the best baseline on Legal by +8.5 points (41.9 vs. 33.4), and disabling post-hoc compaction (41.1) still ranks first by a wide margin. We have included this complete control study in Appendix L.1 (Table 15).

Thank you again for your thorough and insightful review. These cross-provider evaluations and controls have substantially strengthened our empirical validation.

[1] arXiv:2306.05685, Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena

[2] arXiv:2404.04475, Length-Controlled AlpacaEval: A Simple Way to Debias Automatic Evaluators

Dear Reviewer Kyms,

As the discussion period draws to a close, we wanted to gently check in to see whether our responses and the revised manuscript have addressed your concerns. Following your suggestions, we have added cross-provider re-evaluation with gpt-6-luna alongside deterministic-metric comparisons (Section 5, Appendix J), the full breakdown across all 19 sub-benchmarks (Appendix K), and the complete set of length-calibration and baseline-compaction controls (Appendix L.1).

If these new results and clarifications have resolved your questions, we would be very grateful if you would consider raising your score. Please let us know if anything remains unclear, and thank you again for your time and thoughtful feedback!

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Revisions

Dear Area Chairs and Reviewers,

Thank you very much for your time and constructive feedback, and we especially thank Reviewer VaGZ for the multi-round follow-up discussion confirming that their concerns have been addressed. For ease of reference, we briefly summarize our new experiments, clarifications, and blue-highlighted manuscript revisions below:

  1. Judge Independence & Full Task Coverage (Reviewer Kyms, Reviewer ebaR): Clarified judge-backbone family overlap in Sec. 5 and showed that SkillWeave significantly beats the strongest baseline across all backbones on purely deterministic sub-benchmarks with zero LLM-judge involvement (App. J.1, Table 10). Re-scoring all saved predictions with a cross-provider judge (gpt-6-luna) confirms that SkillWeave ranks first across every MemoryBench domain and backbone with strong cross-judge ranking agreement (App. J.2, Table 11). Full results across all 19 sub-benchmarks are in App. K (Tables 12 to 14).
  2. Length Calibration & Negative-Trigger Controls (Reviewer Kyms, Reviewer ebaR): Controlled comparisons show that per-skill adaptive percentiles outperform unconditional, random, and fixed-length compression while triggering on under 6% of items, and applying identical compaction to baselines yields no positive gain, ruling out extra-call self-refinement (App. L.1, Table 15). Retraining with frozen or omitted negative triggers and cross-backbone hard-filter ablations further confirm the necessity of co-evolving soft exclusions (App. L.2, Table 16).
  3. Shared-KB, 2×22\times 2 Grouping, & Context-Budget Parity (Reviewer VaGZ, Reviewer ebaR): Sharing the exact frozen KB with AutoSkill confirms that SkillWeave retains a substantial lead across domains at under one-third total cost (App. L.3, Table 17). A full 3-seed 2×22\times 2 grouping ×\times naming control shows that coherent semantic grouping is required for explicit topic names to anchor distinct genres without inflating BM25 false matches (App. L.4, Table 18). Under a strict KB retrieval budget matched to SimpleMem's median context, SkillWeave continues to lead on both dialogue-memory tasks and Open-Domain overall (App. M.1, Table 19).
  4. Related-Work Positioning & SkillOpt Call Accounting (Reviewer ebaR, Reviewer VaGZ): Sharpened Secs. 1 & 2.2 and added App. I (Table 9) to distinguish MemSkill's reward-driven memory operations and position SkillWeave alongside MSCE and Mem²Evolve across five dimensions. Clarified Sec. 6.3 and added the exact build-call breakdown for SkillOpt, showing that its overhead stems primarily from candidate validation gating rather than update granularity (App. M.2, Table 20).

Thank you again for helping us strengthen our work.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.