acceptodds
Under review as a conference paper at ICLR 2027

SkillMuse: Multi-Turn User Simulation as a Feedback Generator for Skill Evolution

Abstract

In enterprise conversational settings such as customer support, LLM-based assistants rely on high-quality domain skills to answer; yet no mechanism attributes execution failures to skill defects and turns them into targeted improvements—operational data keeps accumulating while skill quality stagnates. Recent work on skill self-evolution builds a revision loop toward closing this gap, but the feedback driving the loop is derived from single-turn question-answering evaluation: a single exchange exposes only the gaps in the user's opening statement, so once the first revision has patched them, the evaluation supplies little further signal, and defects that surface only across multiple turns remain unreached—the loop is intact, yet evolution stalls. Governance in these systems likewise rests on a scalar score, which can only reject a degraded candidate outright, without diagnosing or repairing its structural cause. We argue that the binding constraint on sustained skill evolution lies less in editing capability or the number of iterations than in whether the evaluation feedback keeps supplying trustworthy evolution gradients. We introduce SkillMuse, coupling two components. Trustworthy feedback generates the gradient: we recast multi-turn user simulation from an evaluation endpoint into a feedback generator, so follow-up questions expose defects layer by layer and every round of revision both consumes and produces feedback. Controllable governance constrains the gradient direction: an independent governance layer actively repairs factual degradation and structural bloat after each revision, so degradation does not accumulate across rounds. Across six categories of cloud services, nine production skills, and 98 skill-reference files, SkillMuse exceeds self-reflection-based evolution by 23.0 points and single-turn-QA-driven evolution by 15.4 points.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
Large Language Model Agents, Agent Skill Evolution, Multi-Turn User Simulation, Self-Evolving Systems

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_DWfpOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies skill self-evolution for LLM customer-support agents, arguing that single-turn QA feedback stops exposing new defects after the first revision. The authors propose SkillMuse, which reconstructs a constrained simulated user from each human-handled ticket (intent agenda, behavior facts, emotion trajectory, gated by an intent state machine), verifies multi-turn trajectories against the human solution, forwards only failures attributed to "Knowledge Gap" to a bounded skill editor, and adds a governance layer that checks fact consistency against the initial and previous skill and flags structural bloat or broken references. On nine production skills from one cloud provider, SkillMuse reaches 81.8% TSR after four rounds versus 66.4% for SkillForge-style single-turn evolution and 58.8% for self-reflectiom.

Critical strengths
  1. The setting with held-out and chronologically split evaluation is realistic and reasonable. The paper evolves nine production skills, including 98 reference files and 2,000 escalated tickets, splits tickets chronologically into development and evaluation sets, and selects versions on the development set only. This is reasonable because skill-evolution work is often evaluated on the tickets it was evolved on; a temporal split is the right protocol for a deployment claim.

  2. The user simulator is built from real tickets and checked by humans. Each simulated user is rebuilt from a real human-handled ticket, including the user's requests, known facts, and emotion changes, and an intent state machine ends the dialogue only after all requests have been raised and answered. Two domain experts blindly compared 200 simulated dialogues with the real tickets and reported 95.3% agreement. This is useful because the whole method depends on whether the simulated user behaves like a real one, and this check gives direct human evidence for it instead of relying only on the LLM judge.

Critical weaknesses
  1. The major concern is that the evaluation protocol may favour SkillMuse by construction, and the comparison appears to be a single run. TSR on the evaluation set is computed by running the same multi-turn User Agent and the same Verifier that SkillMuse uses to generate its feedback on the development set. SkillMuse is therefore optimized against the evaluator used at test time, whereas the SkillForge baseline is optimized with single-turn QA and then judged in the multi-turn protocol. Part of the 15.4-point gap may thus reflect a match between training feedback and test protocol rather than better skills. In addition, it is unclear whether the experiments are run with different seeds and whether significance tests are conducted.

  2. Another concern is that the key claim of the paper, i.e., multi-turn dialogue can expose defects that single-turn QA cannot, is not directly verified. The paper does not show how many of the failure signals in later rounds actually come from the second or later turns of a dialogue. In fact, in the paper's own case study, the error is already visible in the first exchange. Since the main difference from SkillForge is the feedback source, and using a user simulator to train dialogue systems is not new, this claim is what the novelty mainly relies on. Without such an analysis, SkillMuse looks more like SkillForge with a multi-turn simulator plugged in. Besides, the ablation study only has two variants, and other components, e.g., attribution filtering, the intent state machine, and the dual-anchor check, are not tested separately. The regression rate is only reported with governance, so it is hard to tell how much governance actually helps. The paper also claims that the evolution is faster and cheaper than manual curation, but no token or time cost is reported. The authors are encouraged to address these issues.

Questions for authors
  1. Have the authors evaluated all methods with a different simulator or judge, or with human evaluation, instead of the same simulator and Verifier used to generate SkillMuse's feedback?
  2. Are the results in Table 2 obtained from a single run? Have the authors run the experiments with different seeds?
Reviewer_Vb7QOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

SkillMuse uses a simulated multi-turn conversation to identify failures. It then classifies these failure to determine if the skills deficiencies can be repaired. During the evolution process it adds factual and structural checks to control skill degradation. Paper also reports substantial improvements over baseline.

Critical strengths
  1. The paper addresses the important problem of skill degradation in existing methods. The method was tested on real-world production data which shows the practical use of the method.
  2. Classifying failures into different types is a useful approach to reducing noisy feedback. The reported reduction in skill growth is also a promising result supporting SkillMuse's approach of controlling bloat.
Critical weaknesses
  1. The paper compares SkillMuse to existing methods that use single-turn feedback, but does not evaluate how these methods perform when given the same multi-turn input. Section 4.3 shows that single turn QA regresses the results and compared to other baselines shows no improvement. This makes it difficult to justify the need for other components.

  2. The method relies heavily on LLMs for every step in the pipeline. The paper does not evaluate the accuracy of failure attribution, LLM errors at each step may make the evolution noisy.

  3. L389-395 claims that follow-up questions and clarifications expose defects layer by layer. The paper only shows increasing TSR and does not analyze whether new defects were identified. The case study in Appendix D doesn't support the claim.

Questions for authors
  1. L247-269 How is the ground truth relevant skill determined? The method assigns an immediate failure when the correct skill is not activated, but what if the agent activates the correct skill later in the conversation?
  2. It would be interesting to see a comparison of the types of edits made by SkillForge and SkillMuse during skill evolution.
  3. Samples with incomplete intent coverage are excluded from evaluation. Are these samples also excluded when evaluating the other baselines?
  4. Could the authors provide comparisons with other baselines in terms of cross-round regression rate and knowledge bloat?
  5. What is the justification for choosing 60 as the threshold for considering a ticket successfully resolved?
  6. The released code appears to provide only a small set of example tickets. I independently executed the released code using gpt-5.6-sol for four evolution cycles. I saw mixed results with Cloudforge improving, Code Assistant dropping and Palworld remaining constant. Although my model choice differs from that used in the paper, these results raise concern over the evaluation stability. Could the authors report confidence intervals for the results?
Reviewer_RTCqOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces SkillMuse, which uses multi-turn user simulations to improve textual skills for customer-support agents. It separates simulator coverage from agent performance, attributes failures, and constrains subsequent skill edits. On 2,000 real support tickets covering nine skills, with a chronological development/evaluation split, SkillMuse reaches 81.8% task success after four rounds, compared with 66.4% for single-turn feedback and 58.8% for self-reflection.

Critical strengths
  • This paper addresses a practical problem using real support tickets and a chronological held-out evaluation across nine production skills.

  • The feedback-source ablation keeps attribution, editing, and governance fixed, providing useful evidence for the benefit of multi-turn simulation.

Critical weaknesses
  • The main gains come from simulated conversations. Human checks of simulator fidelity and verifier agreement do not establish that the evolved skills resolve real customer interactions more successfully.

  • Table 6 reports declining regression only for the full system. Without a corresponding no-governance comparison, the claim that governance preserves existing capabilities is not established.

Questions for authors
  • Do the simulated gains translate into higher resolution rates in the reported production deployment?

  • Can you compare regression rates with and without governance on the same evaluation cases?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.