acceptodds
Under review as a conference paper at ICLR 2027

MSCPM: Multi-level Skill Composition for Procedural Memory in LLM Agents

Abstract

Currently, considerable research on LLM-based agents distills procedural memory from the past trajectories of agents to avoid re-exploring recurring tasks and enhance decision-making, yet existing methods remain limited in skill composition. Inspired by human procedural memory, we propose Multi-level Skill Composition for Procedural Memory (MSCPM), a framework that organizes memory through asymmetric inter-skill relations and explicitly models multi-level skill composition. Specifically, we extract operation, subgoal, and task-domain skills bottom-up through clustering and chunking, forming a hierarchical skill graph. On this graph, a cascade of coarse-to-fine subgoal selection, typed plan orchestration, and dynamic operation assembly composes skills into execution strategies tailored to each new task. Across AppWorld and BFCL-V3, MSCPM surpasses the strongest baseline by 1.94–2.58/1.99–3.97 points in Avg@4/Pass@4, and improves over the no-memory baseline by 26.1%/18.0% and 14.7%/13.7% with Qwen3-32B, rising to 35.8%/26.4% and 22.5%/23.7% with Qwen3-8B. Smaller agents equipped with MSCPM even surpass larger memory-free ones.

Primary area:
foundation or frontier models, including LLMs
Keywords:
LLM agents, agent memory, procedural memory, skills

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 2 reviews, above 85% of the others.
Reviewer_LsfXOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

MSCPM (Multi-level Skill Composition for Procedural Memory) is a training-free procedural-memory framework for LLM agents, motivated by two analogies to human procedural memory: hierarchical chunking and dynamic co-activation. It builds a three-layer hierarchical skill graph bottom-up — operation, subgoal, and task-domain skills — via LLM annotation, embedding-based clustering, and PrefixSpan frequent-pattern chunking, storing only downward cross-layer edges. At inference, a three-stage cascade composes skills per task: coarse-to-fine subgoal selection (locate task domains → expand to candidate subgoals → LLM-select), typed plan orchestration (each plan step tagged Inherit/Adapt/Create), and dynamic operation assembly (co-activation edges among sibling operations induced transiently and ranked by personalized PageRank). A skill-iteration mechanism uses seeded constrained clustering to absorb or create nodes across batches, and Appendix C proves the memory bank's size is upper-bounded by the number of underlying patterns KH (not by trajectory count) under an ε-separability assumption. On AppWorld and BFCL-V3 with Qwen3-8B/14B/32B, MSCPM beats the strongest baseline (among AWM, ReMe, GoS, SkillX) by 1.94–2.58 Avg@4 / 1.99–3.97 Pass@4, with larger relative gains on smaller backbones, and smaller MSCPM-equipped agents reportedly surpass larger memory-free ones. Ablations attribute most of the gain to the hierarchy and to dynamic assembly.

Critical strengths
  1. Strong, honest comparison against the right baselines — this is the main thing the other procedural-memory submissions in this area lack. Unlike papers that compare only against their own memoryless ablation, MSCPM evaluates against four representative, current competitors spanning the exact taxonomy it critiques: flat (AWM), insight-rewriting (ReMe), graph (GoS), and hierarchical (SkillX). It reports standard deviations over three runs, uses two benchmarks with genuinely different structure (AppWorld's 457 APIs and train–test shift vs. BFCL-V3's narrow tool set), and the gains are consistent across all three backbone sizes. The ablation story is also internally coherent and falsifiable: the authors predict that the task-domain layer and subgoal selector should help more on AppWorld than BFCL-V3 (because BFCL exposes only 10–30 tools) and the data confirm it (−1.58/−0.84 on AppWorld vs −0.38/−0.05 on BFCL). An ablation whose asymmetry matches a stated mechanism is more convincing than uniform drops.

  2. The "transient within-layer co-activation" idea is a genuine design contribution with a matching efficiency argument. The central novelty — storing only cross-layer edges and inducing within-layer relations on demand via shared-parent siblings, then dissolving them after the task — is a clean answer to the real tension the related work identifies (hierarchical graphs under-compose within a layer; full graphs over-connect and are seed-sensitive). It's not just asserted: the growth proof in Appendix C gives the storage claim teeth by bounding node count by pattern diversity KH rather than trajectory count, and the skill-iteration curves (Fig. 2f–j) empirically show new-node creation decaying to absorption. This is a more principled treatment of memory-bank growth than most of this literature offers.

Critical weaknesses
  1. The absolute performance is low enough that the headline framing ("small + MSCPM beats large memory-free") rests on a handful of points near the noise floor, and the pipeline's cost is entirely unreported. On AppWorld, even the best 32B result is 34.52% Avg@4 — these are hard tasks where all methods are failing most of the time, so the 1.94–2.58-point margin over SkillX, while consistent, is small in absolute terms and the std bands overlap in several cells (e.g., 8B AppWorld: MSCPM 19.40±0.17 vs SkillX 16.52±0.68 is clear, but 14B BFCL MSCPM 56.94±0.54 vs SkillX 54.67±0.73 is ~2 points with ~0.7 std — tight but not overwhelming). The "smaller-beats-larger" claim is the kind of statement that needs a significance test, not just non-overlapping means read by eye. More seriously, MSCPM is by far the heaviest pipeline here — six LLM modules offline (annotators, inducers) plus three LLM-driven composition stages online (selector, orchestrator, PageRank assembly) — yet the paper reports no token cost, latency, or construction cost anywhere, on either axis. A framework whose selling point over flat methods is "graph flexibility at near-hierarchical cost" cannot omit the cost numbers that would substantiate "near-hierarchical." Without them, a reader cannot tell whether the 2-point gain is bought with 2× the inference calls of AWM.

  2. The convergence proof is real but largely decorative — it proves a property no one disputes, under an assumption that the paper's own clustering is designed to violate. Appendix C proves the node count is bounded by KH and new-node creation decays. But this follows almost immediately from ε-separability (Assumption 1): once you assume same-pattern annotations are <ε apart and cross-pattern ≥ε apart, "clusters map injectively to patterns" is nearly definitional, and the decay is a standard missing-mass argument. The assumption is also exactly what fails in practice — LLM-annotated skill names exhibit the "surface-form variation" the paper cites as its reason for clustering, which means same-pattern items are often not within ε and the clean bound degrades to the c-factor and η-fraction extensions the authors relegate to a paragraph. So the proof certifies bounded growth in the regime where clustering is trivial, and says little about the realistic regime. This isn't fatal, but the paper leans on "provably convergent growth" as a headline contribution, and the theorem is weaker than that billing suggests.

Questions for authors
  1. Can you report construction cost (tokens/LLM calls for the six offline modules) and per-task inference cost (tokens, latency, LLM calls for the three composition stages) for MSCPM versus AWM, ReMe, GoS, and SkillX? The "near-hierarchical cost" claim is central and currently unsupported.

  2. For the small-margin cells and the "smaller+MSCPM > larger memory-free" claim, can you run a paired significance test across the three runs? Specifically: is 8B-MSCPM vs 14B-No-memory significant on each benchmark?

  3. How sensitive is ε-separability to real LLM annotation noise? Can you report the empirical η (fraction of cross-pattern mis-clusters) and show how node count and performance behave as the per-layer thresholds ε_op/ε_sg vary around the chosen 0.23/0.28?

  4. The whole pipeline depends on Qwen-produced skill annotations. Did you evaluate annotation/clustering quality directly (e.g., human-judged cluster purity on a subset), and how do downstream gains degrade when annotations are noisier?

  5. Retrieval is "permitted only once at the beginning of each task" for all baselines — but MSCPM's dynamic assembly induces co-activation graphs per plan step during composition. Is that composition counted as a single up-front retrieval, and does this give MSCPM more effective memory access than the baselines under the stated fairness constraint?

  6. BFCL-V3's training split is 50 randomly chosen tasks; AppWorld uses 90. How stable are the results to the BFCL split choice, given you created it yourselves?

7 comments
AuthorsOpenReview ↗

Response to Reviewer LsfX (1/7) W1 & Q1: Offline memory-construction / online execution cost

W1 & Q1: Offline memory-construction / online execution cost

Response. We have added a new subsection, §4.4 Cost Analysis (see Figure 2 in the manuscript), specifically to analyze the offline memory-construction cost and the online execution cost. We compare the offline memory-construction and online execution costs of all methods on AppWorld and BFCL-V3 with Qwen3-32B.

Offline memory-construction cost covers LLM calls, token usage, and wall-clock time averaged per iteration round over three iteration rounds. As shown in Figure 2(a)–(c) and (g)–(i), AWM summarizes the successful trajectories of multiple similar tasks into an abstract workflow with LLMabstract\mathrm{LLM}_{\mathrm{abstract}}, achieving the lowest construction cost. ReMe extracts experiences via LLMsuccess\mathrm{LLM}_{\mathrm{success}}, LLMfailure\mathrm{LLM}_{\mathrm{failure}}, and LLMcontrastive\mathrm{LLM}_{\mathrm{contrastive}}, and verifies each with LLMvalidator\mathrm{LLM}_{\mathrm{validator}}. SkillX extracts plan, functional, and atomic skills via LLMplan\mathrm{LLM}_{\mathrm{plan}}, LLMfunctional\mathrm{LLM}_{\mathrm{functional}}, LLMatomic\mathrm{LLM}_{\mathrm{atomic}}, LLMfeedback\mathrm{LLM}_{\mathrm{feedback}}, and LLMvalidator\mathrm{LLM}_{\mathrm{validator}}. On BFCL-V3, where all APIs are exposed in the system prompt, atomic extraction is unnecessary. For GoS, we feed the option skills extracted by the MSCPM pipeline as the source for building its skill graph, and additionally employ LLMdiscriminator\mathrm{LLM}_{\mathrm{discriminator}} to annotate its edges with four types of relations between relevant skills (dependency, workflow, semantic, and alternative) together with confidence scores. The offline construction cost of MSCPM is higher than that of AWM, yet lower than or comparable to the strong baselines ReMe and SkillX on most metrics, and lower than that of GoS in all cases.

Online execution cost covers the same metrics averaged per task trajectory. As shown in Figure 2(d)–(f) and (j)–(l), all memory-augmented methods reduce the number of online interaction steps compared with the memory-free method, indicating that memory augmentation reduces redundant exploration and execution. AWM's compact workflows consume few tokens and little time, yet it takes the most interaction steps among memory-augmented methods and achieves the lowest success rate, showing that purely abstract workflows cannot effectively guide agent execution. ReMe additionally applies LLMrerank\mathrm{LLM}_{\mathrm{rerank}} and LLMrewrite\mathrm{LLM}_{\mathrm{rewrite}} to the retrieved memories, incurring extra online cost. SkillX rewrites the retrieved memories into a plan via LLMplan-rewrite\mathrm{LLM}_{\mathrm{plan\text{-}rewrite}} and retrieves low-level skills for each plan step, further invoking LLMrerank\mathrm{LLM}_{\mathrm{rerank}} when many candidates are returned, likewise incurring a high online cost. MSCPM achieves the fewest LLM calls among all methods, together with lower wall-clock time and token usage than the strong baselines SkillX and ReMe.

Summary. MSCPM achieves offline memory-construction and online execution costs lower than those of the strong baselines while improving task success rates, demonstrating the efficiency of our method.

AuthorsOpenReview ↗

Response to Reviewer LsfX (2/7) W1 & Q2: Statistical significance of the improvements

Response. We complement the two conclusions, namely (i) the improvement of MSCPM over the strongest baselines and (ii) a smaller model augmented with MSCPM outperforming larger memory-free models, with a significance analysis based on McNemar's exact test.

For each comparison, we pair the binary outcomes (success/failure) of the two systems on each individual (task, run, execution). Let bb denote the number of pairs where MSCPM succeeds while the baseline fails, and cc the number of pairs where MSCPM fails while the baseline succeeds. After discarding pairs on which the two systems agree, let n=b+cn=b+c be the total number of discordant pairs. The exact two-sided pp-value is

p=min⁡!(1,;;2∑k=0min⁡(b,c)(nk)(12)n).p = \min\\!\left(1,\\;\\; 2 \sum_{k=0}^{\min(b,c)} \binom{n}{k} \left(\frac{1}{2}\right)^{n}\right).

The following Table A reports the improvement Δ (in percentage points) between MSCPM and the strongest baseline in each cell of Table 1, as well as the corresponding p-value. For Avg@4 cells, binary outcomes are paired per (task, run, execution) (3 runs × 4 executions per task; 2016 pairs on AppWorld and 1800 on BFCL-V3). For Pass@4 cells, pass indicators are paired per (task, run) (3 runs per task; 504 and 450 pairs in total).

Table A: Significance of MSCPM over the strongest baseline in each cell of Table 1.

ModelAppWorld Avg@4 Δ\Delta (pp)AppWorld Pass@4 Δ\Delta (pp)BFCL-V3 Avg@4 Δ\Delta (pp)BFCL-V3 Pass@4 Δ\Delta (pp)
Qwen3-8B+2.58 (0.0237)+3.97 (0.0339)+2.34 (0.0273)+3.78 (0.0009)
GLM-4-Z1-9B+3.32 (0.0118)+4.17 (0.0086)+2.50 (0.0312)+3.33 (0.0107)
Qwen3-14B+2.58 (0.0212)+3.38 (0.0430)+2.27 (0.0396)+3.11 (0.0451)
Qwen3-32B+2.33 (0.0495)+1.99 (0.0987)+1.94 (0.0410)+2.22 (0.0768)

Out of the 16 comparisons, 14 are statistically significant (p<0.05p < 0.05). The only two exceptions are the two Pass@4 cells of Qwen3-32B (p=0.0987p=0.0987 and p=0.0768p=0.0768), for two reasons: (1) the improvement of this model is the smallest among all models (+1.99/+2.22), because the largest model has the strongest intrinsic capability and thus benefits least from memory augmentation, i.e., the memory-augmentation effect diminishes as the underlying model becomes stronger; (2) the pairing granularity of Pass@4 is the coarsest (only 3 pass indicators per task), which limits the statistical power. Moreover, the direction of improvement is perfectly consistent across all 16 comparisons, and having 14 out of 16 simultaneous tests significant is far beyond the chance level.

Table B: Smaller models augmented with MSCPM vs. larger memory-free models (paired in the same way as above). "--" indicates that the smaller MSCPM-augmented model does not exceed the larger memory-free model in that cell.

ComparisonAppWorld Avg@4 Δ\Delta (pp)AppWorld Pass@4 Δ\Delta (pp)BFCL-V3 Avg@4 Δ\Delta (pp)BFCL-V3 Pass@4 Δ\Delta (pp)
Qwen3-8B + MSCPM vs. GLM-4-Z1-9B (no mem.)+3.92 (0.0022)+6.75 (0.0002)+5.73 (0.0013)+10.89 (0.0001)
Qwen3-8B + MSCPM vs. Qwen3-14B (no mem.)----+1.06 (0.1102)+5.11 (0.0029)
GLM-4-Z1-9B + MSCPM vs. Qwen3-14B (no mem.)--+0.60 (0.3750)+3.06 (0.0235)+6.89 (0.0014)
Qwen3-14B + MSCPM vs. Qwen3-32B (no mem.)+0.89 (0.4421)--+2.94 (0.0449)+6.66 (0.0003)

As shown in Table B, compared with slightly larger memory-free models, the smaller MSCPM-augmented models win comprehensively and significantly (Qwen3-8B + MSCPM vs. GLM-4-Z1-9B: all four cells with p<0.005p < 0.005). Compared with substantially larger models, almost all cells show significant advantages on BFCL-V3 (e.g., all Pass@4 cells with p<0.003p < 0.003), presumably because the training and test distributions of BFCL are closer to each other, so the memory augmentation transfers better (see our response to Q6 below for an analysis of the BFCL-V3 split); on AppWorld, whose task distribution is more complex, the smaller models approach but do not consistently exceed the larger ones. Where cells are not significant, we frame them as the boundary of the claim rather than as counterexamples. The revised text states that smaller agents equipped with MSCPM can surpass larger memory-free ones — an existence claim established by the significant cells above. This boundary also follows from the mechanism: memory augmentation contributes a roughly constant gain (+5–9 Avg@4 and +8–14 Pass@4 over each backbone's no-memory score, Table 1), whereas the scale gap to a memory-free model grows (+8.3 for 8B→14B and +13.1 for 8B→32B on AppWorld). Significance therefore recedes exactly as the scale gap grows — the dose–response pattern expected of a bounded memory gain; conversely, an 8B model with memory beating a 32B one in every cell would be implausible. We do not claim superiority in the cells where the scale gap exceeds the memory gain.

AuthorsOpenReview ↗

Response to Reviewer LsfX (3/7) W2: Positioning and scope of the growth analysis

W2: Positioning and scope of the growth analysis

Response.

(1). We agree with the assessment and have repositioned the analysis. Under strict ϵ\epsilon-separability, the pattern–node correspondence does follow from the update rule, and the decay calculation is a standard missing-mass argument; the proposition is an idealized coverage analysis, not an unconditional guarantee for the full pipeline. We have restructured Appendix C accordingly and reworded the "provably convergent growth" phrasing in the conclusion to "bounded, coverage-driven growth".

(2). One clarification on surface-form variation. Surface-form variation is a property of annotation text, whereas Assumption 1 concerns the embedding geometry: differently worded annotations of the same skill can still lie within ϵ\epsilon of each other in embedding space — this is precisely why the pipeline clusters embeddings rather than strings. Nevertheless, we agree that strict global separability is not established for LLM-generated annotations and should not be presented as a practical guarantee.

(3). Revisions and empirical support (see Q3 and Q4): growth stability does not require semantic clustering correctness. The critique conflates two questions that Assumption 1 answers simultaneously — whether clusters are semantically correct, and whether the memory grows without bound — and our original presentation did not separate them either. Appendix C now separates them: (i) a new geometric capacity bound (no semantic assumptions): with a frozen encoder and frozen index embeddings, retained node indices form an ϵ\epsilon-packing of the embedding space, so ∣V(m)∣≤Pϵ(X)|V^{(m)}| \le P_\epsilon(\mathcal{X}), finite for any fixed ϵ\epsilon — this holds exactly in the regime the reviewer describes, where same-pattern annotations fragment and distinct patterns merge (Pϵ(X)P_\epsilon(\mathcal{X}) is a capacity bound at the threshold's resolution, not a semantic-quality guarantee); (ii) a new decay bound without separability: under i.i.d. annotation batches and no pruning, the uncovered embedding mass UmU_m is monotonically non-increasing, E[Δm∣Fm]≤b,Um\mathbb{E}[\Delta_m \mid \mathfrak{F}_m] \le b\\,U_m, and E[Δm]≤b(Pϵ(X)−∣V(0)∣)/m→0\mathbb{E}[\Delta_m] \le b(P_\epsilon(\mathcal{X}) - |V^{(0)}|)/m \to 0; (iii) the original KHK_{\mathcal{H}} result is retained as the idealized semantic special case (KH≤Pϵ(X)K_{\mathcal{H}} \le P_\epsilon(\mathcal{X}) under Assumption 1). We have also corrected several imprecisions in the original statements: the creation bound is now stated in conditional form E[Δm∣Fm]≤bMm\mathbb{E}[\Delta_m \mid \mathfrak{F}_m] \le b M_m (the previously stated unconditional form with the random MmM_m on the right-hand side was ill-posed; averaging gives E[Δm]≤b,E[Mm]\mathbb{E}[\Delta_m] \le b\\,\mathbb{E}[M_m]); "independent of the number of rounds and trajectories" now refers to the upper bound rather than the node count; per-round monotone decay is claimed only under a fixed annotation batch size; and the (c,η)(c, \eta) extensions are restated with their correct hypotheses (the multiplicity cap cc must be global across rounds; an η\eta-fraction of mis-annotation preserves bounded growth only if the errors are not persistently novel — recurring errors add at most finitely many nodes, while persistently novel ones accumulate but remain capped by the geometric capacity bound). Empirically, practice operates close to the analyzed regime: the negative-silhouette fraction is 1–6% across layers and benchmarks (Q3); node counts decrease monotonically as ϵ\epsilon grows, as the packing capacity predicts; node creation decays monotonically across construction rounds (Figure 3(f)–(h)); and under annotation-level noise injection up to p=0.3p=0.3, downstream gains are largely retained (Q4: ≥71.8\geq 71.8\\%/75.1\\% at the most damaging injection point) — the injected annotations reuse existing (name, desc) pairs and hence recur rather than being persistently novel, the benign regime of the corrected η\eta-extension.

Reviewer_ozLrOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes MSCPM, a procedural memory framework that organizes past experience into three levels: operations, subgoals, and task domains. For a new task, the agent selects relevant subgoals, decides which skills to reuse, adapt, or create, and assembles the operations needed for execution. Experiments on AppWorld and a multi-turn subset of BFCL-V3 show consistent improvements over existing memory methods across three Qwen3 model sizes.

Critical strengths

MSCPM outperforms the compared methods on both benchmarks with 8B, 14B, and 32B models. The gains over the strongest baselines are modest, but their consistency and the reporting of variation across runs make the results more convincing. Separating operations from subgoals and task domains gives the agent a way to connect high-level planning with concrete actions. The ablations support the value of hierarchy, with removing it causing the largest performance drop on both benchmarks.

Critical weaknesses

(1) The benefit of the full architecture is not sufficiently established. The method combines several mechanisms, but the improvements over the strongest baselines are around 2-3pp in Avg@4. Some components contribute very little in particular settings, e.g., removing the subgoal selector reduces BFCL-V3 performance by only 0.05 points. The paper supports using a hierarchy, but gives less evidence that this particular three-level design and all its components are needed.

(2) The evaluation leaves questions about generality and baseline fairness. The experiments use one model family, AppWorld test-normal, and a limited multi-turn subset of BFCL-V3. All baselines are also restricted to retrieving experience once at the beginning of a task. It is unclear whether this restriction disadvantages methods that could benefit from retrieving memory during execution.

Questions for authors

(1) What evidence supports the three-level design over a simpler operation-subgoal hierarchy? Given the small effect of some components, can a simpler configuration achieve comparable performance with lower cost?

(2) Why is experience retrieval restricted to once at the beginning of each task?

(3) How do memory construction cost, online token usage, and latency compare with the strongest baselines? This would help assess whether the performance gains justify the additional complexity.

3 comments
AuthorsOpenReview ↗

Response to Reviewer ozLr (1/3)W1 & Q1: Necessity of the full three-level architecture

W1 & Q1: Necessity of the full three-level architecture

We thank the reviewer for the valuable comments. Our responses are provided below.

Response.

(1) On “the evidence of the three-level design over a simpler operation-subgoal hierarchy”. In §4.3 Ablation Studies (Table 2), we ablate the three-layer skill graph structure in a top-down manner: removing the task-domain layer first (ablation (d)), and on this basis further removing the subgoal layer (ablation (e), i.e., w/o Hierarchy, operation layer only). Since the skill composition mechanism during online execution is designed on top of the three-layer skill graph structure:

  • after the task-domain layer is removed, coarse-to-fine skill selection is no longer possible;
  • after the subgoal layer is further removed, typed orchestration of subgoals is no longer possible;

the skill composition mechanism becomes incomplete, which ultimately degrades performance (see the excerpt of Table 2 below: once the task-domain layer is removed, coarse-to-fine Selection is disabled; once the subgoal layer is further removed, typed Orchestration is disabled as well). Table 2 (excerpt): mechanism configurations and Avg@4 (%)

VariantSelectionOrchestrationAssemblyAppWorld Avg@4BFCL-V3 Avg@4
MSCPM (full)✓✓✓34.52 ± 0.3961.94 ± 0.10
(d) w/o Task-Domain Layer✗✓✓32.94 ± 0.31 (−1.58 pp)61.56 ± 0.19 (−0.38 pp)
(e) w/o Hierarchy (op. only)✗✗✓30.56 ± 0.09 (−3.96 pp)58.94 ± 0.26 (−3.00 pp)

(2) On "some components contribute very little in particular settings". This phenomenon mainly appears in ablations (a) w/o subgoal selector and (d) w/o task-domain layer on BFCL-V3. We argue that this is primarily attributable to the design of the benchmarks themselves:

  • Task complexity. As shown in Table 1 and Table 2, the same model achieves a lower success rate on AppWorld, indicating higher task complexity.
  • Action-space scale. AppWorld comprises 457 APIs, all of which remain available for the agent to invoke in every task, yielding a large action space; BFCL-V3 comprises only 129 APIs, of which merely 10–30 are actually usable in each task, making the environment relatively constrained.
  • API discoverability. AppWorld does not reveal the full set of available APIs at the start of a task; instead, the agent has to reason step by step during execution and actively explore and query APIs. In contrast, BFCL-V3 places the information of all available APIs in the system prompt from the beginning, so the agent has complete knowledge of every usable API before execution begins.

Therefore, compared with BFCL-V3's simpler tasks and constrained environment, AppWorld relies more on the task-domain layer to help the agent search more thoroughly in a complex action space, and on the subgoal selector to filter out irrelevant skill information. This coarse-to-fine skill composition strategy, which first broadens the search scope and then progressively narrows it down, effectively enhances planning ability. It follows that when the skill library scales up and the action space becomes more complex and unknown, the gains brought by the task-domain layer and the subgoal selector will become more pronounced. This also explains why the performance drops of ablations (d) and (e) are larger on AppWorld than on BFCL-V3.

AuthorsOpenReview ↗

Response to Reviewer ozLr (2/3) W2 & Q2: Experimental protocol and evaluation scope

W2 & Q2: Experimental protocol and evaluation scope

Response.

(1) Why is experience retrieved only once at the beginning of a task? This setting ensures a fair comparison and follows the native protocols of AWM, ReMe, SkillX, and GoS.Except this, we imposed no additional restrictions on any method. Retrieving only at the start of a task reduces the computational overhead of online execution and is also consistent with practical application scenarios, where the agent can access relevant experience before execution begins, thereby enhancing its planning and decision-making. We acknowledge that there is indeed a line of work on retrieving experience adaptively during execution, which is orthogonal to the focus of this paper; however, such online retrieval typically introduces additional computation and time costs.

(2) Why is only a single model family used? Our method is not tied to any specific model family. We have added the results of GLM-4-Z1-9B to Table 1 in the manuscript (see the table below); due to the limited rebuttal period, we started with this one additional model family and will further validate on more model families in future work.

Main results of GLM-4-Z1-9B (newly added to Table 1 in the manuscript)

MethodAppWorld Avg@4AppWorld Pass@4BFCL-V3 Avg@4BFCL-V3 Pass@4
No-memory15.48 ± 0.5233.13 ± 1.5043.33 ± 1.0162.22 ± 1.68
AWM17.86 ± 0.7935.91 ± 1.5045.50 ± 0.4466.44 ± 1.02
ReMe18.80 ± 0.6737.10 ± 0.9147.89 ± 0.5168.22 ± 1.02
GoS18.45 ± 0.6836.90 ± 1.1947.94 ± 0.6969.11 ± 1.02
SkillX19.05 ± 0.3037.30 ± 0.9148.56 ± 0.4271.56 ± 0.39
MSCPM (ours)22.37 ± 0.3441.47 ± 0.9151.06 ± 0.3574.89 ± 0.77

Under this backbone, MSCPM again ranks first in all four cells, surpassing the strongest baseline SkillX by 3.32/4.17 points (Avg@4/Pass@4) on AppWorld and 2.50/3.33 on BFCL-V3. This confirms that the effectiveness of MSCPM is not tied to a specific model family. Moreover, GLM-4-Z1-9B equipped with MSCPM outperforms the same-size-class Qwen3-8B equipped with MSCPM in all four cells. It even exceeds the larger memory-free Qwen3-14B on three of the four cells, namely +3.06 Avg@4 and +6.89 Pass@4 on BFCL-V3 and +0.60 Pass@4 on AppWorld. These results further demonstrate the advantage of MSCPM in amplifying the capability of small models.

(3) Why is only a limited multi-turn subset of BFCL-V3 used? The multi_turn_base is the standard multi-turn tool-calling category of BFCL-V3, and it targets exactly the kind of recurring multi-step tool-calling patterns that procedural memory is meant to transfer. The other categories of BFCL-V3 are unrelated to the research problem studied in this work, i.e., distilling skills from trajectories and composing them. The following table lists all the categories in BFCL-V3:

CategoryCapability tested
multi_turn_baseStandard multi-turn tool calling (used in this work)
multi_turn_miss_paramWhether the agent proactively asks for clarification when required parameters are missing
multi_turn_miss_funcWhether the agent recognizes insufficient tools and informs the user instead of hallucinating
multi_turn_long_contextRobustness to interference in long contexts

AuthorsOpenReview ↗

Response to Reviewer ozLr (3/3) Q3: Offline memory-construction / online execution cost

Q3: Offline memory-construction / online execution cost

Response. We have added a new subsection, §4.4 Cost Analysis (see Figure 2 in the manuscript), specifically to analyze the offline memory-construction cost and the online execution cost. We compare the offline memory-construction and online execution costs of all methods on AppWorld and BFCL-V3 with Qwen3-32B.

Offline memory-construction cost covers LLM calls, token usage, and wall-clock time averaged per iteration round over three iteration rounds. As shown in Figure 2(a)–(c) and (g)–(i), AWM summarizes the successful trajectories of multiple similar tasks into an abstract workflow with LLMabstract\mathrm{LLM}_{\mathrm{abstract}}, achieving the lowest construction cost. ReMe extracts experiences via LLMsuccess\mathrm{LLM}_{\mathrm{success}}, LLMfailure\mathrm{LLM}_{\mathrm{failure}}, and LLMcontrastive\mathrm{LLM}_{\mathrm{contrastive}}, and verifies each with LLMvalidator\mathrm{LLM}_{\mathrm{validator}}. SkillX extracts plan, functional, and atomic skills via LLMplan\mathrm{LLM}_{\mathrm{plan}}, LLMfunctional\mathrm{LLM}_{\mathrm{functional}}, LLMatomic\mathrm{LLM}_{\mathrm{atomic}}, LLMfeedback\mathrm{LLM}_{\mathrm{feedback}}, and LLMvalidator\mathrm{LLM}_{\mathrm{validator}}. On BFCL-V3, where all APIs are exposed in the system prompt, atomic extraction is unnecessary. For GoS, we feed the option skills extracted by the MSCPM pipeline as the source for building its skill graph, and additionally employ LLMdiscriminator\mathrm{LLM}_{\mathrm{discriminator}} to annotate its edges with four types of relations between relevant skills (dependency, workflow, semantic, and alternative) together with confidence scores. The offline construction cost of MSCPM is higher than that of AWM, yet lower than or comparable to the strong baselines ReMe and SkillX on most metrics, and lower than that of GoS in all cases.

Online execution cost covers the same metrics averaged per task trajectory. As shown in Figure 2(d)–(f) and (j)–(l), all memory-augmented methods reduce the number of online interaction steps compared with the memory-free method, indicating that memory augmentation reduces redundant exploration and execution. AWM's compact workflows consume few tokens and little time, yet it takes the most interaction steps among memory-augmented methods and achieves the lowest success rate, showing that purely abstract workflows cannot effectively guide agent execution. ReMe additionally applies LLMrerank\mathrm{LLM}_{\mathrm{rerank}} and LLMrewrite\mathrm{LLM}_{\mathrm{rewrite}} to the retrieved memories, incurring extra online cost. SkillX rewrites the retrieved memories into a plan via LLMplan-rewrite\mathrm{LLM}_{\mathrm{plan\text{-}rewrite}} and retrieves low-level skills for each plan step, further invoking LLMrerank\mathrm{LLM}_{\mathrm{rerank}} when many candidates are returned, likewise incurring a high online cost. MSCPM achieves the fewest LLM calls among all methods, together with lower wall-clock time and token usage than the strong baselines SkillX and ReMe.

Summary. MSCPM achieves offline memory-construction and online execution costs lower than those of the strong baselines while improving task success rates, demonstrating the efficiency of our method.

We thank you again for your time and valuable comments, which have helped us further improve the paper. We hope our responses address your concerns.

Comments on the paper

5 comments
AuthorsOpenReview ↗

Response to Reviewer 89BB (1/3) W1 & Q1: Are the comparisons budget-matched?

W1 & Q1: Are the comparisons budget-matched?

We thank the reviewer for the valuable comments. Our responses are provided below.

Response.

(1) Budget setup. Apart from No-memory, all methods undergo the same three-round iterative process of experience distillation and memory construction, and all three rounds use the same LLM under the same thinking mode. Except for the first round, which starts from shared trajectories (produced by the No-memory method), the trajectories used for memory extraction in each subsequent round are generated by each memory-augmented method itself.

(2) Additional results and clarification.

  • We have added a chart showing task-evolution performance (Figure 3(a)) in §4.5 Skill Iteration Analysis, comparing how the success rates of SkillX and MSCPM evolve across iteration rounds. As shown, MSCPM outperforms SkillX in every round, indicating that the performance gain of MSCPM does not merely come from the evolution in rounds 2–3.
  • We have supplemented Table 2 with standard deviations.
AuthorsOpenReview ↗

Response to Reviewer 89BB (2/3) W2:Offline memory-construction / online execution cost

W2:Offline memory-construction / online execution cost

Response. We have added a new subsection, §4.4 Cost Analysis (see Figure 2 in the manuscript), specifically to analyze the offline memory-construction cost and the online execution cost. We compare the offline memory-construction and online execution costs of all methods on AppWorld and BFCL-V3 with Qwen3-32B.

Offline memory-construction cost covers LLM calls, token usage, and wall-clock time averaged per iteration round over three iteration rounds. As shown in Figure 2(a)–(c) and (g)–(i), AWM summarizes the successful trajectories of multiple similar tasks into an abstract workflow with LLMabstract\mathrm{LLM}_{\mathrm{abstract}}, achieving the lowest construction cost. ReMe extracts experiences via LLMsuccess\mathrm{LLM}_{\mathrm{success}}, LLMfailure\mathrm{LLM}_{\mathrm{failure}}, and LLMcontrastive\mathrm{LLM}_{\mathrm{contrastive}}, and verifies each with LLMvalidator\mathrm{LLM}_{\mathrm{validator}}. SkillX extracts plan, functional, and atomic skills via LLMplan\mathrm{LLM}_{\mathrm{plan}}, LLMfunctional\mathrm{LLM}_{\mathrm{functional}}, LLMatomic\mathrm{LLM}_{\mathrm{atomic}}, LLMfeedback\mathrm{LLM}_{\mathrm{feedback}}, and LLMvalidator\mathrm{LLM}_{\mathrm{validator}}. On BFCL-V3, where all APIs are exposed in the system prompt, atomic extraction is unnecessary. For GoS, we feed the option skills extracted by the MSCPM pipeline as the source for building its skill graph, and additionally employ LLMdiscriminator\mathrm{LLM}_{\mathrm{discriminator}} to annotate its edges with four types of relations between relevant skills (dependency, workflow, semantic, and alternative) together with confidence scores. The offline construction cost of MSCPM is higher than that of AWM, yet lower than or comparable to the strong baselines ReMe and SkillX on most metrics, and lower than that of GoS in all cases.

Online execution cost covers the same metrics averaged per task trajectory. As shown in Figure 2(d)–(f) and (j)–(l), all memory-augmented methods reduce the number of online interaction steps compared with the memory-free method, indicating that memory augmentation reduces redundant exploration and execution. AWM's compact workflows consume few tokens and little time, yet it takes the most interaction steps among memory-augmented methods and achieves the lowest success rate, showing that purely abstract workflows cannot effectively guide agent execution. ReMe additionally applies LLMrerank\mathrm{LLM}_{\mathrm{rerank}} and LLMrewrite\mathrm{LLM}_{\mathrm{rewrite}} to the retrieved memories, incurring extra online cost. SkillX rewrites the retrieved memories into a plan via LLMplan-rewrite\mathrm{LLM}_{\mathrm{plan\text{-}rewrite}} and retrieves low-level skills for each plan step, further invoking LLMrerank\mathrm{LLM}_{\mathrm{rerank}} when many candidates are returned, likewise incurring a high online cost. MSCPM achieves the fewest LLM calls among all methods, together with lower wall-clock time and token usage than the strong baselines SkillX and ReMe.

Summary. MSCPM achieves offline memory-construction and online execution costs lower than those of the strong baselines while improving task success rates, demonstrating the efficiency of our method.

AuthorsOpenReview ↗

Response to Reviewer 89BB (3/3) Q2: (1) Data split and (2) selection of clustering thresholds

Q2: (1) Data split and (2) selection of clustering thresholds

Response.

(1) Train/test split of BFCL-V3. The BFCL split is fixed across all methods and all runs. We randomly split the 200 tasks into a training set and a test set, without any manual curation. We will also release the task-split list in our code for reproducibility.

(2) Selection of clustering thresholds. The clustering thresholds are determined solely from the first-round training trajectories, in a fully unsupervised manner. Specifically, for each layer, we cluster the first-round skill annotations under different candidate thresholds and evaluate the clustering quality with the silhouette coefficient. For any annotation ii in the embedding space, let a(i)a(i) denote the average distance from ii to the other annotations within the same skill node (intra-cluster cohesion), and b(i)b(i) the minimum, over all other nodes, of the average distance from ii to the annotations of that node (separation from the nearest neighboring node). The silhouette coefficient of ii is

s(i)=b(i)−a(i)max⁡a(i),,b(i)∈[−1,1],s(i)=\frac{b(i)-a(i)}{\max\\{a(i),\\,b(i)\\}}\in[-1,1],

where the distance is consistent with the one used for clustering at that layer (cosine distance for the operation and subgoal layers; the mixed distance dmixd_{\mathrm{mix}} with weight α\alpha for the task-domain layer). An s(i)s(i) close to 1 indicates that the annotation is much closer to its own node than to any other node; a value close to 0 indicates that it lies on a node boundary; a negative value suggests a likely misassignment. In the tables: #Nodes is the number of skill nodes after clustering; Silhouette is the average silhouette coefficient over all annotations; Neg.-sil. is the fraction of annotations with a negative silhouette coefficient.

For each benchmark, we first sweep ϵop\epsilon_{op}, then sweep ϵsg\epsilon_{sg} under the optimal ϵop\epsilon_{op}, and finally sweep the (ϵdm,α)(\epsilon_{dm},\alpha) grid, taking the value with the highest average silhouette coefficient at each step (bold in the tables). This yields ϵop=ϵsg=0.28\epsilon_{op}=\epsilon_{sg}=0.28, ϵdm=0.35\epsilon_{dm}=0.35, and α=0.4\alpha=0.4 on BFCL-V3, and ϵop=ϵsg=0.23\epsilon_{op}=\epsilon_{sg}=0.23, ϵdm=0.40\epsilon_{dm}=0.40, and α=0.6\alpha=0.6 on AppWorld. We have included the results below in Appendix D.2.

Operation layer ϵop\epsilon_{op} (BFCL-V3)

ϵop\epsilon_{op}0.200.230.250.280.300.330.35
#Nodes12011710389805838
Silhouette0.320.360.410.440.390.240.20
Neg.-sil.0.000.010.030.030.110.110.20

Operation layer ϵop\epsilon_{op} (AppWorld)

ϵop\epsilon_{op}0.200.230.250.280.300.330.35
#Nodes22919118016213111389
Silhouette0.330.400.380.360.300.220.21
Neg.-sil.0.040.060.100.120.140.190.21

Under the optimal operation-layer threshold, we further conduct similar experiments for the subgoal-layer threshold:

Subgoal layer ϵsg\epsilon_{sg} (BFCL-V3)

ϵsg\epsilon_{sg}0.200.230.250.280.300.330.35
#Nodes42403937332118
Silhouette0.460.490.530.540.510.370.29
Neg.-sil.0.000.000.020.020.030.040.10

Subgoal layer ϵsg\epsilon_{sg} (AppWorld)

ϵsg\epsilon_{sg}0.200.230.250.280.300.330.35
#Nodes57444238322321
Silhouette0.500.520.510.490.360.280.25
Neg.-sil.0.000.010.020.040.070.110.19

Furthermore, we run a grid search over the task-domain-layer threshold ϵdm\epsilon_{dm} and the semantic-distance weight α\alpha (values in the tables are average silhouette coefficients):

Task-domain layer (ϵdm,α)(\epsilon_{dm}, \alpha) (BFCL-V3)

ϵdm\α\epsilon_{dm} \backslash \alpha0.30.40.50.60.7
0.250.490.550.530.510.45
0.300.540.670.560.550.44
0.350.610.690.570.520.47
0.400.500.600.640.530.53
0.450.470.620.650.540.49

Task-domain layer (ϵdm,α)(\epsilon_{dm}, \alpha) (AppWorld)

ϵdm\α\epsilon_{dm} \backslash \alpha0.30.40.50.60.7
0.250.440.490.520.470.40
0.300.490.500.490.510.55
0.350.520.520.510.590.54
0.400.550.580.600.610.60
0.450.470.600.530.560.57

We thank you again for your time and valuable comments, which have helped us further improve the paper. We hope our responses address your concerns.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.