acceptodds
Under review as a conference paper at ICLR 2027

PROSE: Compact and Traceable Skill Evolution

Abstract

LLM agents increasingly use skill documents to reuse experience across related tasks, but automatically maintaining these documents remains challenging. A fully autonomous loop must identify lessons that apply beyond a single task, incorporate them without uncontrolled growth, and retain an inspectable basis for each revision. We introduce PROSE (PROcedural Skill Evolution), an experience-driven method for training the skill document of a frozen agent. Inspired by group-relative optimization, PROSE contrasts repeated attempts of the same task for task-conditioned attribution, then uses a batch-level optimizer to consolidate recurring mechanisms across tasks. It turns these lessons into localized updates to a structured skill document, with programmatic evidence and budget checks that bound document growth and keep each change traceable to recorded experience. Starting from an empty document, PROSE improves held-out success by to points across SpreadsheetBench, AppWorld, and LiveMathematicianBench. It attains the highest mean on every reported metric among the evaluated skill-learning methods. It also produces the most context-efficient skill documents, 43–88% smaller than those of competing methods. On AppWorld, this also holds when a single open-weight 27B model fills every role. We will release our implementation as open-source code. These results support skill documents as a compact and traceable adaptation layer for frozen agents.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
LLM agents, skill learning, experience-driven learning, procedural memory, self-improving agents, prompt optimization, context efficiency, traceability

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_KaXsOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

PROSE learns a skill document for a frozen LLM agent. It compares repeated attempts of the same task, collects lessons supported by multiple tasks, and applies structured edits after checking their supporting evidence. The paper reports results on three benchmarks, with higher mean success rates and shorter documents than the evaluated baselines. It also evaluates a setup using one open-weight model for all roles.

Critical strengths

S1. The learned skills are substantially shorter, using 43–88% fewer characters than the competing methods while achieving higher mean success rates. This reduces the context needed each time the skill is used.

S2. The method also works with one open-weight model serving all roles. With Qwen3.6-27B on AppWorld test-normal, TGC improves from 79.2% without a skill to 91.9%, and SGC from 62.5% to 83.3%.

Critical weaknesses

W1. The methodological contribution is incremental. PROSE combines grouped sampling, trajectory reflection, cross-task consolidation, and constrained document editing. Its main addition is an explicit protocol linking evidence to individual edits and checking whether each edit meets predefined requirements. This improves control and inspectability, but the contribution remains primarily a system-level integration of existing skill-learning ideas.

W2. The ablations do not isolate the contribution of evidence licensing. The free-editing variant removes structured editing and licensing together, while the no-mixed-group-contrast variant excludes entire mixed groups from the evidence available to the optimizer. These comparisons support the complete design, but cannot separate better evidence use from more conservative editing or access to additional useful trajectories.

W3. Comparisons with additional skill extraction and optimization methods, such as SkillCAT and SkillAdam, are needed to establish the method's relative advantages.

Questions for authors

Q1. The free-editing ablation removes structured editing and evidence licensing together. The performance drop therefore cannot be attributed to evidence licensing alone, leaving its independent contribution unclear.

Q2. Additional evaluation on a web-interaction or code-maintenance benchmark would help establish whether the compactness and performance gains extend to other agent workflows.

Q3. Please include additional skill extraction and optimization baselines.

2 comments

Thank you for the detailed response and additional ablation. My remaining concern is that the methodological contribution is still incremental, as the framework mainly integrates existing contrast and structured-editing ideas under a rule-based evidence check whose isolated accuracy gain is not statistically significant in the new ablation, so I will maintain my current assessment.

Thank you for reading the response and the new ablation. One clarification for the record.

We agree, and stated in our response, that removing evidence licensing alone does not change accuracy significantly. We do not claim an accuracy gain for licensing. Its measured effect is on the document: without it the document is 38% larger (3,894 against 2,830 characters) and holds 13 entries instead of 8, because single-task lessons are written in instead of being held pending.

This matches the paper's claim, which is accuracy on par with the strongest baseline with a document 1.8–8.6× smaller in which every entry is traceable to verified evidence. Licensing contributes the compactness and the traceability, at no cost in accuracy.

We appreciate your engagement throughout the discussion.

Reviewer_zfK3OpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper introduces PROSE as a skill evolution method for LLM agents. PROSE borrows the principle of GRPO to perform generation, distillation, and consolidation mechanism. Extensive experiments on SpreadsheetBench, APPWorld, and LiveMathematicianBench demonstrate the effectiveness of the proposed method.

Critical strengths
  • This paper investigates an important and practical research problem.
Critical weaknesses
  • The technical novelty of this paper is quite limited. Generate and Distill can be traced to the series of works like TF-GRPO, Skill-Pro, etc, whereas Update is an overly engineered heuristic design. The gains from the delicate design of the Update step is also unclear, which is not supported by the ablation study. It is more likely that the validation set based gating mechanism plays a major role.

  • The empirical analyses are weak. The adopted baselines are outdated ones, which are more like prompt engineering methods rather than widely recognized skills that are a set of artifacts under a single folder. Furthermore, I suggest the authors following the experimental setups in Trace2Skill to test the generalization of the learned skills; otherwise, it is unknown whether the gains mainly come from overfitting or real generalization. Besides, I am confused why the main experiments use two different kinds of LLMs (GPT-5.6-Luna, DeepSeek-V4-Pro). Please unify all the roles with one single LLM for fair comparison (just as what Table 2 did).

  • The randomness from skill learning is not handled in the experiments. The reported results are just averaged from multiple runs under the same learned skill, which may lead to the concerns on cherry-picked learned skills, especially considering that the performance gains are quite minor (+0.3 SR with 1.1 std in SpreadsheetBench, +0.6 TGC with 0.3 std in APPWorld test-normal).

Questions for authors

See Weakness

Reviewer_5gZgOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes PROSE, improving frozen LLM agents by learning a better skill document. For each task, PROSE samples multiple rollouts, extracts same-task success/failure contrasts or recurring failure patterns, consolidates candidate lessons across tasks, and proposes typed add/edit/move/drop operations on a structured skill state. Host-side checks verify cited trajectory spans and outcomes, enforce evidence and budget constraints, and maintain provenance records. Experiments on SpreadsheetBench, AppWorld, and LiveMathBench report substantial gains.

Critical strengths
  1. The paper presents a concrete and well-specified protocol for skill evolution. In particular, separating LLM-generated proposals from host-side evidence licensing, typed document operations, and versioned provenance is a useful systems contribution. The end-to-end example of an accepted rule revision makes the intended traceability mechanism clear.

  2. The empirical evaluation is broader than a single benchmark: it includes three task families, an unseen-app challenge split, comparisons under a self-hosted 27B model stack, and ablations. The gains over the no-skill condition are substantial, suggesting that the overall idea of learning an external skill artifact from experience is effective.

Critical weaknesses
  1. The strongest empirical claim is not supported by the reported significance tests. Although PROSE has the highest mean in Table 1, Table 5 reports no statistically significant difference from the strongest baseline, SkillOpt, on any of the six main comparisons (e.g., SpreadsheetBench: +0.4, (p=.754); AppWorld normal SGC: +4.2, (p=.102); LiveMathBench:+4.3, (p=.084)). The evidence supports strong improvement over no-skill, but not a robust claim that PROSE outperforms SkillOpt.

  2. Each method appears to be trained only once per benchmark/model setting. The three trials are evaluation repetitions of one learned artifact, not independent end-to-end training runs. Since the paper reports 17–18% task-level nondeterminism, evaluation-level tests over tasks do not establish the variance of the full skill-learning procedure.

  3. The free-editing ablation jointly removes typed state, typed operations, and evidence licensing, so it does not isolate which component drives the effect. The compactness comparison is also confounded by PROSE’s explicit length constraint while the LLM-generated baseline has no comparable constraint.

Questions for authors
  1. Can the authors report results over multiple independent end-to-end training seeds for PROSE and SkillOpt, including variation from rollout sampling, extraction, consolidation, and document updates? Would the claimed advantage over SkillOpt remain statistically significant under this analysis?

  2. Given that none of the Table 5 comparisons against SkillOpt is significant at the conventional 0.05 threshold, please also provide confidence intervals or an equivalent uncertainty analysis for the method-level comparison.

  3. Can the authors compare all methods under a matched worker-visible skill-length or context budget? This would clarify whether PROSE’s compactness and performance advantage persist when baselines receive the same document budget.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response: Summary of Revisions

We sincerely thank all reviewers for their careful and constructive feedback. Based on these suggestions, we have revised the manuscript. The major updates are summarized below.

  1. Restated main claim (Reviewers 5gZg, zfK3). The paper no longer claims the highest mean. The abstract, introduction, Section 4.2 and conclusion now state that PROSE matches the accuracy of the strongest baseline, SkillOpt, with skill documents 1.8–8.6× smaller (Table 3) in which every entry is traceable to verified evidence.

    • Confidence intervals: We added 95% bootstrap confidence intervals for the paired comparisons (Tables 6 and 7, Reviewer 5gZg). Every PROSE − SkillOpt interval in Table 6 includes zero, which is why we report the two methods as on par in accuracy.
  2. Independent training seeds (Reviewers 5gZg, zfK3). We trained three independent seeds of both PROSE and SkillOpt on AppWorld with Qwen3.6-27B in every role (Section 4.2, App. C.3, Table 10). The two methods are on par in accuracy across seeds, every seed of either method improves on the worker without a skill, and PROSE's document is 2.1–4.8× smaller in every pairing of seeds.

  3. Ablations that separate the components of Update (Reviewers 5gZg, zfK3, KaXs). Section 4.3 is rewritten around three variants (Table 4, App. C.2):

    • No evidence licensing (new): Evidence licensing is removed while the structured document and typed operations are kept. The document grows by more than a third, and free editing roughly doubles it; mean accuracy is slightly lower in both variants, though not significantly. Under the frontier-model stack, free editing lowers accuracy significantly (Table 9).
    • Length bounds lifted (new): Raising both length bounds to 100,000 tokens does not enlarge the document, so the compactness comes from the update mechanism and not from a cap (Reviewer 5gZg). This run is also the comparison with no length limit on any method, where PROSE's document remains the smallest (App. C.2).
    • Ablations focused on the Update step: In a mixed group, the same-task pairing and the evidence come from the same rollouts, so the contrast cannot be switched off without also removing information (Reviewer KaXs). Section 4.3 therefore focuses on the components that can be separated cleanly, and no longer includes the variant without mixed-group contrast.
  4. Generalization beyond the training setting (Reviewers zfK3, KaXs). In addition to the unseen-app results on AppWorld test-challenge (Table 1), we added two transfer settings that follow Trace2Skill's setup: SpreadsheetBench documents applied without further training to WikiTableQuestions and HiTab (App. C.5, Table 12), and AppWorld documents moved between the two workers (App. C.4, Table 11).

  5. Clarification on validation (Reviewer zfK3). We clarified that validation scores are recorded for monitoring and do not gate commits, and that the revert safeguard never fired in any reported run (App. A.2). We also report how the pending ledger is used: in the main AppWorld run, 4 of the 10 lessons held pending were later cited by an accepted operation (App. B.2).

  6. Related work and future work (Reviewers zfK3, KaXs). We added Skill-Pro and SkillAdam to the related-work discussion (Section 2), updated the venues of published baselines in the references, and list skill artifacts that include scripts and reference files as future work (Section 5).

In the revised manuscript, these updates are highlighted in blue for your convenience.

We hope the revision resolves your concerns, and we look forward to further discussion.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.