acceptodds
Under review as a conference paper at ICLR 2027

Rethinking Agent Skill Orchestration: Do we Need to Compile Skills Before Each Execution?

Abstract

As agent skills become widely adopted, the prevailing practice is to retrieve relevant skills for each task and orchestrate them on the fly. When an agent repeatedly solves tasks of the same type, however, recompilng skills for every instance incurs extra LLM calls and latency, and discards the memory accumulated in earlier executions. Prior work compiles skills into dependency graphs or induces reusable skills from trajectories offline, yet still leaves orchestration to test time, performed anew for each task. We observe that successful executions of the same task type follow a highly consistent composition, so composition can be done once, at induction time. Building on this observation, we propose FlatSkill, which compiles an agent's skill-guided trajectories into one prefab per task type, pairing a slotted, task-level procedure with rules verified against the trajectories. At test time, the agent only selects the matching prefab and fills in its slots. On ALFWorld, ScienceWorld, WebShop and MedCalc-Bench, FlatSkill outperforms existing methods in all evaluation settings with DeepSeek-V3.2, while taking fewer interaction steps than the strongest baseline. Prefabs compiled by a stronger model can also be reused directly by a smaller one, demonstrating the benefit of amortizing skill composition across executions rather than repeating it for every task.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Agent, skills, skill orchestration, procedural memory, offline compilation

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_Z9CXOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper organizes the execution records of similar tasks into reusable text procedures. When a new task arrives, the corresponding procedure is selected, the current task information is filled in, and then it is handed over to the agent for execution. The authors compared the performance and execution costs of tasks on multiple environments and models, and also tested whether the procedures organized by the strong model could help the small model.

Critical strengths

S1. The method has shown improvements in multiple environments and models, and some settings have also reduced the number of execution steps.

S2. The procedures organized by a strong model can directly assist a small model, and in most test settings, they are more effective than the procedures organized by the small model itself. This indicates that these experiences can be reused across models without the need for retraining the executing model.

S3. The comparison of equal-length irrelevant prompts is useful, indicating that the benefits are not solely due to longer prompts. The authors also included a simple but effective control that directly retrieves successful records.

Critical weaknesses

W1. I think the added contribution of the method is rather limited. My initial reaction was, isn't this just a workflow? "AWM" can already generate reusable text workflows from execution experience, and "Agentic Plan Caching" also saves and reuses plan templates. This article emphasizes preparing a complete procedure in advance for each type of task, but it does not directly compare this with similar methods. It is not clear what benefits this approach brings over the existing workflow reuse.

W2. The appendix provides estimates for compilation costs and break-even points, but it does not clarify whether collecting execution records, retrying failed tasks, and verifying modifications are all included. If these costs are not accounted for, it is unclear how many tasks would be needed to recover the cost when establishing a procedure library from scratch.

W3. Figure 1 compares each execution record with the most similar one, but finding similar examples does not mean that the entire type of task can share a single procedure. Even within the same type of task, even if there are several different approaches, each record may still find very similar neighbors. It is not clear whether preparing a procedure in advance is still more effective than per-task orchestration when the differences in these approaches are significant.

Questions for authors

See weaknesses.

Can you compare the small model's performance when using the original successful execution records of the strong model and when using the procedure derived from these records, under similar context budgets? I want to see if converting these trajectories into a workflow provides any additional value for the small model, or if directly providing the strong model’s successful execution trajectories can achieve a similar effect.

6 comments
AuthorsOpenReview ↗

Response to Reviewer Z9CX – Part I

Thank you for your careful review and for recognizing FlatSkill’s improvements across environments, cross-model reuse, and controlled ablations. We address your concerns below.

W1: Thank you for highlighting the connections to AWM and Agentic Plan Caching. We appreciate the opportunity to clarify FlatSkill’s contribution and provide direct comparisons with these methods.

Our core comparison GraSP, also builds on skill-augmented trajectories, but retrieves and composes skills on the fly for each task instance. FlatSkill consolidates the realized action compositions across multiple instances of the same task type into a prefab before inference. A task type may require multiple skills; FlatSkill makes their realized composition reusable as a unit through routing and slot binding, avoiding repeated skill retrieval and composition for subsequent instances.

FlatSkill’s central contribution is to make task-type-level skill composition a reusable unit. It consolidates the action compositions realized in multiple skill-augmented trajectories into a prefab that can serve subsequent instances of the same task type. The distinction concerns what is compiled, how executions are consolidated, and what computation remains at test time.

  • Compared with AWM: AWM induces reusable subroutines from execution experience and provides them as procedural memory. FlatSkill targets the task-level composition realized by a skill-augmented agent: multiple skills may contribute to one execution, and their resulting action composition is consolidated across instances into one prefab per task type. Its procedure is selected according to coverage of the observed execution backbones and supplemented with outcome-verified rules.
  • Compared with APC: APC extracts generalized plan templates from completed Planner–Actor executions and uses a lightweight planner to adapt cached templates during subsequent executions. FlatSkill consolidates multiple executions of a task type before inference. At test time, it routes the task to a prefab and binds its slots; an unmodified ReAct agent then executes with this guidance, without retrieving or recomposing the original skills.

To directly evaluate the benefit of this design, we added comparisons with AWM and APC, using the same source trajectories across all methods in the comparison. Entries report reward / average interaction steps.

MethodALFWorld SeenALFWorld UnseenWebShop SeenScienceWorld SeenScienceWorld UnseenMedCalc Seen
GraSP80.6 / 14.583.6 / 14.846.2 / 17.884.9 / 11.981.3 / 12.578.7 / 4.3
APC76.7 / 15.388.9 / 13.250.8 / 19.771.7 / 17.572.5 / 16.188.9 / 2.3
AWM83.6 / 15.091.0 / 12.949.4 / 20.484.6 / 12.582.4 / 14.089.6 / 2.3
FlatSkill98.6 / 9.194.0 / 10.961.7 / 17.486.3 / 13.979.4 / 15.591.9 / 2.8

FlatSkill achieves higher reward than all baseline methods. On ALFWorld Seen and WebShop, its reward gains over GraSP are 18.0 and 15.5 points, and its gains over AWM are 15.0 and 12.3 points. Across both ALFWorld splits and WebShop, it also uses fewer interaction steps than all three comparators. These results demonstrate the value of consolidating skill-augmented experience into a reusable task-type composition.

AuthorsOpenReview ↗

Response to Reviewer Z9CX – Part II

W2: Thank you for asking us to clarify the scope of the cost accounting. Appendix C.9 reports the incremental cost of the trajectory-to-prefab compilation calls. The corresponding break-even estimate therefore concerns amortizing this compilation overhead. FlatSkill supports an iterative reuse cycle: an agent starts with ReAct+Skills, compiles its collected execution experience into prefabs, and can subsequently compile updated prefabs from new prefab-augmented trajectories. Section 4.6 already evaluates this cycle: a second round of compilation improves ScienceWorld reward from 86.3 to 89.0 on the seen split and from 79.4 to 84.4 on the unseen split. This allows accumulated experience to improve a reusable task-level composition across rounds. Other skill orchestration Methods such as GraSP use collected trajectories only to guide orchestration for each new task, and FlatSkill retains the resulting composition itself for reuse across subsequent instances.

W3: Thank you for raising this point. Tasks of the same type often share a common sequence of key actions, even when their complete executions differ. FlatSkill extracts these execution backbones from multiple trajectories and compiles the shared steps into a slotted procedure. We then compare successful and failed trajectories to propose outcome-verified rules that explain what the agent should do when particular conditions arise. Together with the branches retained during procedure compilation, these rules help the prefab handle differences between specific tasks. So sharing one prefab does not require every instance to follow an identical execution path. Figure 1 illustrates recurring patterns, while compilation considers multiple trajectories to capture both shared steps and useful conditional behavior. On-the-fly orchestration, like GraSP also uses past trajectories as memory to guide orchestration for the current task. FlatSkill uses execution experience to learn the common procedure and conditional guidance ahead of time, allowing subsequent tasks to reuse this composition while adapting their actions to the environment.

Q: Thank you for suggesting this comparison. We conducted two additional experiments with Qwen3-8B:

  1. The small model directly uses the strong model’s trajectories. We provide Qwen3-8B with up to eight original successful execution records from DeepSeek, retrieved by task-text similarity from the candidate trajectories originally used for procedure compilation for the same task type. We also conduct a comparison under similar context budgets, selecting the single most similar record and match its context budget to the corresponding DeepSeek-compiled prefab. This record contains the original task description, reasoning, actions, and observations, with the skill documents removed from the original system message. When it exceeds the budget, we retain its beginning and end and remove the middle.

  2. The small model compiles the strong model’s trajectories. We ask Qwen3-8B to compile prefabs from DeepSeek’s trajectories and then execute tasks using these prefabs. For comparison, we also include prefabs compiled by Qwen3-8B from its own trajectories and prefabs compiled by DeepSeek from DeepSeek’s trajectories.

Input to Qwen3-8BALFWorld SeenALFWorld UnseenWebShop SeenScienceWorld SeenScienceWorld UnseenMedCalc Seen
DeepSeek trajectories, full79.7 / 13.982.8 / 13.525.9 / 20.975.9 / 15.067.8 / 17.183.5 / 2.6
DeepSeek trajectory, matched token budget28.6 / 25.335.8 / 24.75.3 / 27.660.6 / 18.549.5 / 20.442.8 / 3.5
Prefab: DeepSeek trajectories → Qwen3-8B compilation74.3 / 14.176.9 / 15.024.0 / 21.072.9 / 15.268.6 / 16.365.2 / 6.5
Prefab: Qwen3-8B trajectories → Qwen3-8B compilation81.4 / 12.979.9 / 13.738.9 / 11.676.8 / 15.171.3 / 16.685.6 / 2.0
Prefab: DeepSeek trajectories → DeepSeek compilation86.4 / 12.091.8 / 11.741.7 / 15.377.0 / 14.473.0 / 16.084.2 / 2.0

The additional experiments show that providing the strong model’s trajectories to Qwen3-8B, either to guide execution directly or to compile prefabs, does not improve overall reward over the prefabs compiled by Qwen3-8B from its own trajectories already reported in the main paper. This holds both under comparable context budgets and when up to eight original successful execution records are provided. The DeepSeek-compiled prefabs already evaluated in the paper achieve higher reward with fewer interaction steps in all six settings than either directly supplying these records or asking Qwen3-8B to compile them. These results show that the cross-model benefit comes from transferring the strong model’s compiled prefabs, rather than providing its trajectories alone.

Dear Reviewer, we sincerely appreciate your time and effort in reviewing our manuscript and offering valuable suggestions. As the author-reviewer discussion phase is drawing to a close, we would like to confirm whether our responses have effectively addressed your concerns. We provided detailed responses to your concerns a few days ago, and we hope they have adequately addressed your issues. If you require further clarification or have any additional concerns, please do not hesitate to contact us. We are more than willing to continue our communication with you. Best regards.

Reviewer_LrmsOpenReview ↗
Rating
3
Confidence
4
Informativeness
1
Presentation
3
Soundness
3
Read review
Paper summary

This work explores how we can avoid repeated cost of skill retrieval and compositions by generating "prefabs", which are similar to templates of skill use generated prior to deployment time from offline trajectories.

Critical strengths
  • The approach is practical and can be adopted by in real use cases pretty easily.
  • The approach scores better than the baselines across most of the settings.
  • It is interesting to see reuse of prefabs between different models.
Critical weaknesses
  • Benchmarks evaluated on are fairly simple and the model used is outdated, making the evaluation not very representative of modern LLMs and tasks.
  • The setting of generating skills/memory/trajectory reuse before deployment is not novel. E.g., this is the setting of SkillWeaver [1].

[1] Zheng et al., Skillweaver: Web agents can self-improve by discovering and honing skills, arXiv 2025

Questions for authors

Questions:

  • A core assumption of the paper is that within a "task type", trajectories are highly consistent. Is this true in real user usage of agents beyond curated benchmarks which tend to have very similar tasks? Are you able to provide evidence for this?
5 comments
AuthorsOpenReview ↗

Response to Reviewer Lrms – Part I

Thank you for your encouraging feedback and for recognizing FlatSkill’s practicality, empirical performance, and cross-model prefab reuse. We address your concerns below.

W1: We have expanded our evaluation to four additional agent benchmarks, including DocOps, Terminal-Bench 2.0, FeatureBench Lite, and DeepSWE, using DeepSeek-V4.1-Flash-1017. These experiments extend the original evaluation, including the LHTB software-engineering subset in Appendix C.1, to further settings involving extended tool use, software evolution, and complex software-engineering tasks.

For DocOps, we use the skills provided with the benchmark; for the other three benchmarks, we use the same skill library as in our LHTB evaluation. All methods in this additional evaluation use DeepSeek-V4.1-Flash-1017. Each entry reports reward / average steps.

MethodDocOpsTerminal-Bench 2.0FeatureBench LiteDeepSWE
ReAct13.6 / 143.568.2 / 28.785.3 / 123.971.0 / 92.1
ReAct+Skills25.0 / 136.752.0 / 48.285.4 / 117.373.3 / 87.1
SkillNet18.3 / 50.650.0 / 31.883.6 / 107.273.3 / 76.6
GraSP19.6 / 35.770.8 / 37.883.8 / 118.069.0 / 87.7
FlatSkill26.7 / 37.788.5 / 39.887.4 / 113.276.7 / 89.1

FlatSkill achieves the highest reward on all four additional benchmarks, exceeding the strongest baseline on each benchmark by 1.7, 17.7, 2.0, and 3.4 points. On Terminal-Bench 2.0, FlatSkill achieves 88.5 compared with 70.8 for GraSP. On DocOps, it improves reward over the strongest baseline, ReAct+Skills, while reducing average steps from 136.7 to 37.7—a 72.4% reduction. These results show that the benefits of task-type-level pre-orchestration extend to the updated backbone and the expanded set of agent workloads.

Q1: Thank you for this important question. The regularity targeted by FlatSkill is a shared execution backbone within a recurring task family. Task-specific entities are represented by slots, exploratory detours are abstracted during backbone extraction, and the ReAct agent continues to adapt its actions to observations. This allows the reusable procedure to accommodate variation in inputs and environment states.

Several studies support the practical relevance of this distinction between shared structure and instance-specific variation:

  • Procedural structure in complex tasks. Alibaba Group motivates procedural memory through recurring reasoning patterns, tool sequences, and recovery strategies across complex tasks. Its discussion distinguishes these shared structures from changing execution conditions, including network disruptions, UI changes, and evolving data schemas. This supports identifying reusable procedural structure within varied executions.[1]

  • Reusable plans in data-dependent applications. Agentic Plan Caching describes how data-analysis requests can share a high-level plan while the details depend on the input data. Its financial-analysis example reuses a calculation procedure across different companies and fiscal years. In its FinanceBench experiment, cached plans are reused for 46 of 100 sampled queries, providing concrete evidence of recurring plan structure in this application setting.[2]

These findings support the practical relevance of reusable execution structure. FlatSkill operationalizes this opportunity at the task-type level by consolidating shared compositions before inference, while its slot binding and observation-conditioned execution handle instance-specific variation. The additional evaluations in W1 further demonstrate the utility of this design across a broader range of agent workloads.

  1. Memp: Exploring Agent Procedural Memory. ACL 2026
  2. Agentic Plan Caching: Test-Time Memory for Fast and Cost-Efficient LLM Agents. NeurIPS 2025
AuthorsOpenReview ↗

Response to Reviewer Lrms – Part II

W2: Thank you for highlighting this relevant work. FlatSkill’s contribution is a task-type-level pre-orchestration method, which compilie the composition realized through existing skills into a prefab that can be reused across recurring task instances**. The distinction from SkillWeaver concerns the computational problem being addressed and the role of the resulting artifact:

  1. Skill acquisition and skill orchestration address different stages. SkillWeaver discovers, synthesizes, tests, and refines executable Python APIs through autonomous exploration. FlatSkill starts from an existing skill library and addresses how the composition emerging from its use can be compiled and reused. Its input is the agent’s skill-augmented execution trajectories, from which it extracts the action compositions that successfully complete tasks.

  2. FlatSkill consolidates multiple executions into one prefab per task type. A recurring task type may require multiple skills. FlatSkill aggregates execution backbones from instances of that type and compiles their shared composition into a slotted task-level procedure, together with outcome-verified rules and interface statistics. This makes the task-level composition itself the reusable unit. Procedure candidates are selected according to their coverage of the observed execution backbones.

  3. The compiled prefab directly supports test-time execution. SkillWeaver selects relevant APIs from its library, taking their preconditions into account, and makes them available for the agent to call. FlatSkill instead routes the task to its prefab, binds the task-specific slots, and supplies the result to an unmodified ReAct agent. This moves the recurring composition work before inference while preserving the agent’s ability to respond to environment observations.

In addition, our core comparison GraSP, also builds on skill-augmented trajectories, but retrieves and composes skills on the fly for each task instance. FlatSkill consolidates the realized action compositions across multiple instances of the same task type into a prefab before inference. A task type may require multiple skills; FlatSkill makes their realized composition reusable as a unit through routing and slot binding, avoiding repeated skill retrieval and composition for subsequent instances.

We have also added a comparison with SkillWeaver using the same source trajectories for all three methods. Each entry reports reward / average steps.

MethodALFWorld SeenALFWorld UnseenWebShop SeenScienceWorld SeenScienceWorld UnseenMedCalc Seen
GraSP80.6 / 14.583.6 / 14.846.2 / 17.884.9 / 11.981.3 / 12.578.7 / 4.3
SkillWeaver84.3 / 13.891.0 / 12.359.3 / 16.485.0 / 12.181.7 / 12.789.2 / 2.2
FlatSkill98.6 / 9.194.0 / 10.961.7 / 17.486.3 / 13.979.4 / 15.591.9 / 2.8

FlatSkill achieves the highest reward in five of the six evaluated settings. On ALFWorld Seen, it improves reward over SkillWeaver by 14.3 points while reducing average steps from 13.8 to 9.1. On ALFWorld Unseen, it improves reward by 3.0 points while reducing steps from 12.3 to 10.9. It also achieves higher reward on WebShop, ScienceWorld Seen, and MedCalc. These comparisons support the value of compiling task-type-level compositions alongside the established approaches of skill acquisition and per-task orchestration.

Dear Reviewer, we sincerely appreciate your time and effort in reviewing our manuscript and offering valuable suggestions. As the author-reviewer discussion phase is drawing to a close, we would like to confirm whether our responses have effectively addressed your concerns. We provided detailed responses to your concerns a few days ago, and we hope they have adequately addressed your issues. If you require further clarification or have any additional concerns, please do not hesitate to contact us. We are more than willing to continue our communication with you. Best regards.

Reviewer_wzL9OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

They propose a new technique, FlatSkill, that groups skills by task type so that skill-based agents don’t need to assemble skills fresh for every task. They evaluate the technique with DeepSeek-V3.2, Qwen3-235B and Qwen3-8B, across four benchmarks, ALFWorld, ScienceWorld, WebShop and MedCalc-Bench. FlatSkill achieves the best score in almost all settings and shows better performance than per-task orchestration methods (SkillNet, GraSP) and experience-reuse baselines (Reflexion, ExpeL).

Critical strengths
  • Propose a new technique for skill-based agents
  • FlatSkill outperforms existing methods in all evaluation settings
Critical weaknesses
  • I feel that the four benchmarks, ALFWorld, ScienceWorld, WebShop and MedCalc-Bench, are not really representative for agents. Some of them are old and since coding is one of the main domains for agents, it'd be better to focus on long horizon coding tasks.
  • Their method requires grouping tasks by type. Wondering whether there will be misrouting issues and whether this can influence performance.
  • I think the method overlaps with Agent Workflow Memory, which also induces reusable workflows from past trajectories and provides them to the agent for new tasks. However, the paper doesn't compare their method with AWM
  • It'd be helpful to move related work to main body
Questions for authors

Please see the weaknesses above.

4 comments
AuthorsOpenReview ↗

Response to Reviewer wzL9 – Part I

Thank you for your thoughtful comments and for recognizing the contribution and empirical performance of FlatSkill. We address your concerns point by point below.

W1: We agree that long-horizon coding tasks are an important setting for evaluating agent capabilities. The original submission includes an evaluation on seven software-engineering tasks from LHTB in Appendix C.1. FlatSkill achieves the highest average reward of 35.44, compared with 32.84 for GraSP and 31.99 for SkillNet, while requiring fewer interaction steps: 52.7 versus 58.3 and 55.7, respectively.

We have expanded the evaluation to four additional benchmarks: DocOps, Terminal-Bench 2.0, FeatureBench Lite, and DeepSWE. All methods in this additional evaluation use DeepSeek-V4.1-Flash-1017. For DocOps, we use the skills provided with the benchmark; for the other three benchmarks, we use the same skill library as in our LHTB evaluation. Each entry below reports reward / average steps.

MethodDocOpsTerminal-Bench 2.0FeatureBench LiteDeepSWE
ReAct13.6 / 143.568.2 / 28.785.3 / 123.971.0 / 92.1
ReAct+Skills25.0 / 136.752.0 / 48.285.4 / 117.373.3 / 87.1
SkillNet18.3 / 50.650.0 / 31.883.6 / 107.273.3 / 76.6
GraSP19.6 / 35.770.8 / 37.883.8 / 118.069.0 / 87.7
FlatSkill26.7 / 37.788.5 / 39.887.4 / 113.276.7 / 89.1

FlatSkill achieves the highest reward on all four additional benchmarks, exceeding the strongest baseline on each benchmark by 1.7, 17.7, 2.0, and 3.4 points. The largest gain occurs on Terminal-Bench 2.0, where FlatSkill achieves 88.5 compared with 70.8 for GraSP. On DocOps, FlatSkill improves reward over the strongest baseline, ReAct+Skills, while reducing average steps from 136.7 to 37.7—a 72.4% reduction. Together with the original LHTB results, these evaluations establish the benefits of task-type-level pre-orchestration across a broader range of agent workloads.

W2: Thank you for raising this important question. We directly evaluate routing sensitivity on ScienceWorld using DeepSeek-V4.1-Flash-1017 by routing randomly selected subsets of 25%, 50%, and 100% of tasks to prefabs from other task types within ScienceWorld. We also evaluate a setting in which every task is assigned to its nearest incorrect task type. Each entry reports reward / average steps.

Routing SettingSeen (Reward / Steps)Unseen (Reward / Steps)
Original Routing88.91 / 13.5985.89 / 15.41
25% Random Misrouting87.62 / 14.5985.14 / 15.52
50% Random Misrouting86.27 / 14.7883.83 / 15.33
100% Random Misrouting83.27 / 15.5278.82 / 16.14
100% Nearest Wrong-Type Routing86.05 / 12.6479.18 / 15.09

The original router achieves the highest reward on both splits. Compared with 100% random misrouting, correct routing provides gains of 5.64 points on Seen and 7.07 points on Unseen. It also outperforms routing to the nearest incorrect task type by 2.86 and 6.71 points, respectively. These results demonstrate the importance of matching each task to its appropriate prefab, even when the alternative prefabs belong to the same domain, and support FlatSkill’s task-type-specific organization.

AuthorsOpenReview ↗

Response to Reviewer wzL9 – Part II

W3: Thank you for pointing out this relevant comparison. FlatSkill focuses on amortizing the orchestration of an existing skill library across recurring task instances. Its relationship to AWM is best understood through the compilation target, granularity, and test-time use:

  1. Compiling the composition realized through skill use. A task may require multiple skills. FlatSkill collects skill-augmented trajectories and extracts execution backbones that capture the action compositions realized as the agent uses these skills. It then compiles the shared composition across instances of the same task type. This moves repeated orchestration into a reusable compilation stage, whereas per-task orchestration methods such as GraSP assemble skills for each new task.

  2. Consolidating experience into one prefab per task type. AWM induces reusable subroutines from past trajectories and incorporates them into workflow memory, as described in AWM. FlatSkill uses the task type as its compilation unit: it consolidates multiple executions into a single prefab containing a slotted procedure, outcome-verified rules, interface statistics, and an optional exemplar. Procedure candidates are selected according to their coverage of the observed execution backbones. This directly targets the task-level composition shared across recurring instances.

  3. Reusing the compiled composition through routing and slot binding. At inference time, FlatSkill selects the corresponding prefab, binds its task-specific slots, and supplies it to an unmodified ReAct agent. The agent continues to respond to environment observations, while per-task skill retrieval and recomposition are removed. The compiled prefab is also reusable across models: Figure 4(a) demonstrates that prefabs compiled from a stronger model’s trajectories can be used directly by a smaller model.

To complement these methodological distinctions, we have added a comparison with AWM. All three methods below use the same source trajectories, enabling a direct comparison of how they organize and reuse the available experience. Each entry reports reward / average steps.

MethodALFWorld SeenALFWorld UnseenWebShop SeenScienceWorld SeenScienceWorld UnseenMedCalc Seen
GraSP80.6 / 14.583.6 / 14.846.2 / 17.884.9 / 11.981.3 / 12.578.7 / 4.3
AWM83.6 / 15.091.0 / 12.949.4 / 20.484.6 / 12.582.4 / 14.089.6 / 2.3
FlatSkill98.6 / 9.194.0 / 10.961.7 / 17.486.3 / 13.979.4 / 15.591.9 / 2.8

FlatSkill achieves the highest reward in five of the six evaluated settings. Compared with AWM, it improves reward by 15.0 points on ALFWorld Seen, 3.0 points on ALFWorld Unseen, and 12.3 points on WebShop, while requiring fewer interaction steps in all three settings. These results demonstrate the value of task-type-level pre-orchestration relative to both workflow-memory reuse and per-task skill orchestration.

W4: Thank you for this suggestion. We have moved the Related Work section into the main body of the revised manuscript and will further refine the layout in the camera-ready version.

Dear Reviewer, we sincerely appreciate your time and effort in reviewing our manuscript and offering valuable suggestions. As the author-reviewer discussion phase is drawing to a close, we would like to confirm whether our responses have effectively addressed your concerns. We provided detailed responses to your concerns a few days ago, and we hope they have adequately addressed your issues. If you require further clarification or have any additional concerns, please do not hesitate to contact us. We are more than willing to continue our communication with you. Best regards.

Dear Reviewer, as the author–reviewer discussion period ends tomorrow, we would like to follow up once more on our responses to your comments. If you have had a chance to review them, we would greatly appreciate hearing whether any of your concerns remain unresolved. We would be happy to provide further clarification before the discussion closes. Thank you again for your time and consideration. Best regards.

Comments on the paper

1 comment

Dear PC, SAC, AC, and Reviewers,

We sincerely thank you for your constructive feedback and expert insights, which have substantially strengthened the manuscript. We are encouraged by the reviewers’ recognition of FlatSkill’s clarity, practicality, and effectiveness in improving skill-based agent performance, as well as the value of cross-model prefab reuse.

Based on your feedback, we have made the following updates to the paper:

  1. We added comparisons with other trajectory-reuse methods in Section 5.3 and Table 2, clarifying FlatSkill’s distinct research problem and demonstrating the benefits of task-type-level pre-orchestration through comparisons using the same source trajectories. FlatSkill achieves the highest reward in five of the six settings.
  2. We added experiments with a newer model on four additional benchmarks in Appendix C.2, extending the evaluation to complex, long-horizon agent tasks. FlatSkill achieves the highest reward on all four benchmarks.
  3. We added new experimental results on routing perturbations in Appendix C.4 to demonstrate robustness to misrouting and validate the benefits of task-type-specific prefab selection.
  4. We added new comparisons of direct trajectory reuse and prefab compilation across models in Appendix C.6, demonstrating that prefabs compiled by a stronger model provide more effective guidance to a smaller model than directly supplying the stronger model’s trajectories, including under matched context budgets.
  5. We moved Related Work into Section 2, while retaining the extended discussion in Appendix A, to clarify FlatSkill’s relationship to prior work and improve readability.

Our responses to individual reviewers explain these results and clarifications in detail, and the revised manuscript includes all the updates summarized above. Our responses address the reviewers’ concerns through additional experiments and methodological clarifications. Following our latest responses and follow-ups, no further questions have been raised.

We appreciate the time and effort you have devoted to evaluating our work.

Sincerely,
The Authors of Submission #17864

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.