- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper studies skill construction, reuse, and the programmatic routing problem for skills, and proposes SCRIPT (Skill Composition and Routing via Interpretable Programmed Transitions). The paper reformulates the skill retrieval program as a planning problem using a route program. The proposed method first learns a skill library from expert trajectories. It then uses the skill library to relabel trajectories at the skill level. A coding agent generates a router program from the skill-level trajectories. At deployment, the router returns several skill-related action sequences, and the LLM decides whether to use the skill or an LLM-generated action. In failure mode, the LLM can diagnose and apply skill modification, library modification, and routing modifications to repair. The proposed method is evaluated on Science World and App World. SCRIPT is not only among the best but also uses far fewer tokens than the strongest baselines.
Critical strengths
- The method performs best on the training set and also generalizes well to unseen tasks. The router program's hierarchical structure and low-level skill actions enable this generalization. With the router program, the LLM also does not need to attend to the skill library or select an action from it. The LLM only needs to select the routed result to take actions; as a result, token usage will be much lower than the baselines.
Critical weaknesses
- The core limitation of the method is that it is not scalable. The router program is trained on tens of environments, but it is hard to imagine what it would look like if it learned from hundreds. Figure 4 shows that the method performs better with more data. I'd like to see token usage with more data to show how the method scales.
Questions for authors
- In deployment, the routing program will cooperate with the fallback LLM policy. I want to know how often the fallback policy is used. If the LLM fallback policy handles everything, your method reduces to an LLM agent. You need an action-distribution measurement of routing policies to justify your method.
- How do you get the training trajectories? Are they learned via RL, come from a teacher model, or come from human expert demonstrations? You are trying to learn a unified decision logic into a routing program. The data quality may affect your results. I would like to know whether the method is robust enough to handle mixed trajectory sources. Suppose there are 2 training data sources, A and B; you can learn the routing policies from A, B, and A plus B. Will the learned routing program change dramatically between training data sources?
- You provide an external anonymous code, but it only contains the README file.
- In section 3.3, your method (SCRIPT) minimizes the learning object with a coding agent. The program's search space is rigid and hard to optimize. Do you use an evolutionary search method like AlphaEvolve [1]?
- How is the router program executed? Does the LLM read the router program and observation, then execute it? For example, in Figure 2, there is a boolean function named needs_melting. Does the Science World environment provide this function? Or does an LLM read the router program and observation and give feedback? How is the boolean function implemented and executed? I ask because the created function may introduce additional computational overhead not reflected in token usage.
- Does one environment use a shared router program? If so, scaling may be hard. In the learning stage, the coding LLM will take many more tokens to construct the router program.
[1] Novikov, Alexander, et al. "AlphaEvolve: A coding agent for scientific and algorithmic discovery." arXiv preprint arXiv:2506.13131 (2025).
7 comments
Official Comment II
WORKSPACE DATA one point per skill boundary in a gold trajectory
points[i] = {
"task": "Your task is to boil tin. ...",
"history": [("action: go to kitchen", "observation: You move to the kitchen."), ...],
"skill": "heat-substance" # gold label = expected output
}
INSPECT
agent → workspace for p in points: print(p["history"][-1], "->", p["skill"])
(or any Python: overview(), explain(i), Counter(...), regex tests, ...)
workspace → agent CONTEXT 1 ... observation: a glass cup (containing tin) -> heat-substance
CONTEXT 2 ... observation: a metal pot (containing lead) -> heat-substance
CONTEXT 3 ... observation: You move to the kitchen. -> acquire-thermometer
...
⇒ agent infers the shared condition of each skill
e.g. heat-substance: "the substance is in a container in hand"
DRAFT
agent → workspace def choose(task, history) -> skill | None
workspace → agent verifier: for each point, choose(p["task"], p["history"]) vs p["skill"]
(replays the data above — no LLM, no environment)
accuracy 0.927 MDL score
wrong: gold skill predicted count returned by points
locate-substance navigate-to-room ×5 line 92 7, 20, …
report-result heat-substance ×5 line 78 9, 22, …
REPAIR
agent → workspace edit(old='...', new='...') (or any Python)
workspace → agent edit applied · re-run on ALL points
accuracy 0.927 → 0.964 MDL score
wrong: gold skill predicted count returned by points
report-result heat-substance ×5 line 78 9, 22, …
agent → workspace edit(old=..., new=...) # a broader rule, to fix another group
workspace → agent edit applied · re-run on ALL points
...
SELECT
every version is logged; return the lowest-MDL version with accuracy ≥ 0.5, else the most accurate
(The inspect, draft, and repair stages are summarized from a logged session rather than fixed steps, and the agent decides at every turn what to run next and when to stop.)
Q5. Router execution
The router is a Python function that the interpreter runs directly. It takes the task instruction and execution history as input and returns the next skill ID or None. During router learning, the coding agent writes and revises the code, while a deterministic Python verifier runs it on cached decision points and reports any mismatches. At deployment, the learned router selects skills without an LLM call.
needs_melting in Figure 2 illustrates a helper function that the coding agent writes as part of the router. These helpers are part of the generated Python code and use string matching or regular expressions to extract relevant information from ScienceWorld’s text observations. Before using a helper in a routing rule, the coding agent checks its outputs across all cached training decision points.
W1. Scalability
To examine how programmatic routing scales with additional training data, we added an experiment covering three ScienceWorld task types (use-thermometer, measure-melting-point, and find-animal), with 636 training trajectories in total. We train SCRIPT’s skill libraries and routers using 30, 150, and all 636 trajectories, and compare SCRIPT with ACE and GoS at the 30- and 150-trajectory settings.
| Method | Trajectories | Score | Learning Input tokens | Learning Output tokens | Learning Time |
|---|---|---|---|---|---|
| SCRIPT | 30 | 93.9 | 0.31M | 25k | 3 min |
| SCRIPT | 150 | 92.8 | 1.08M | 88k | 8 min |
| SCRIPT | 636 | 93.9 | 4.35M | 328k | 44 min |
| GoS | 30 | 83.8 | 6.8M | 200k | 15 min |
| GoS | 150 | 79.0 | 34M | 700k | 1.2 h |
| ACE | 30 | 81.1 | 25.7M | 570k | 1.85 h |
| ACE | 150 | 77.5 | 279.4M | 1980k | 11.8h |
The table below reports cumulative learning tokens as SCRIPT processes 636 trajectories in batches of up to 30.
| Trajectories learned | 120 | 240 | 360 | 480 | 600 | 636 |
|---|---|---|---|---|---|---|
| Cumulative tokens | 1.01M | 1.51M | 2.37M | 3.28M | 4.25M | 4.68M |
SCRIPT uses fewer tokens to learn from all 636 trajectories than either ACE or GoS uses to learn from 30. Meanwhile, GoS’s score drops from 83.8 to 79.0 as its training set grows from 30 to 150 trajectories.
Official Comment III
Four aspects of SCRIPT help explain this scaling behavior:
1. Compact, verified code vs. growing skill documents. As the skill library grows, methods that place all skills in context give the LLM longer prompts and more candidates to consider at each decision. In SCRIPT, the coding agent extracts rules shared across decision contexts, and the MDL objective favors compact programs that fit the recorded decisions. At test time, the router selects skills by executing a deterministic Python function without an LLM call.
2. Offline verification vs. repeated environment rollouts. Evaluating policy revisions through task rollouts requires repeated LLM calls and environment interactions. SCRIPT executes each router revision on cached decision points and compares its predictions with the demonstrated skill IDs. The coding agent can therefore test every edit against the full cache without additional LLM calls or environment rollouts during verification.
3. Parallel skill induction vs. sequential accumulation. ACE updates its playbook sequentially as it processes training trajectories. SCRIPT can induce skills from different groups of trajectories in parallel and verify the router offline. In our runs, SCRIPT learned from 636 trajectories in 44 minutes, while ACE took 11.8h to process 150 trajectories.
4. Shared rules across additional examples. SCRIPT turns skill selection into a coding task with automated tests and feedback, letting the coding agent use its ability to find patterns in data, express them as code, and refine them through testing. For common skills, the agent compares their decision contexts with those of easily confused skills to identify when each applies. For rare skills, it inspects the available examples to find the conditions for using them. A single rule can cover many decision points, so additional examples that follow a known pattern require no code changes. When new examples reveal a gap, the agent can revise the relevant rule and test the edit against all cached decisions.
Q1. Fallback usage
Learned skills handle most of the execution actions. On ScienceWorld, the fallback policy accounts for 11.1% of execution on Seen tasks and 17.9% on Unseen tasks, decreasing to 13.8% after evolution (Table 7 in Appendix A.5).
Q2. Source of training trajectories
Following ACE, Trace2Skill, and GoS, we use the gold trajectories provided with each benchmark, so all of these methods learn from identical experience.
To test robustness to the trajectory source, we keep the number of training trajectories per task type fixed and replace half of the gold trajectories with successful trajectories collected by a ReAct agent with the same backbone.
| Score | Success Rate(%) | |
|---|---|---|
| Gold | 87.68 | 80.49 |
| Gold + ReAct | 88.27 | 78.04 |
Replacing half of the gold trajectories with ReAct trajectories changes Score by +0.59 and Success Rate by −2.45 points, indicating that SCRIPT is robust to the trajectory source.
Q3. Code
The full code is now available at the anonymous link.
Q6. Router organization
We organize the routing programs according to the structure of each benchmark. In ScienceWorld, we learn one router per task type because different task types involve distinct goals and skills. In AppWorld, we learn a single router shared across all scenarios because they use the same set of apps and APIs. As discussed in our response to W1, the scaling results show that router learning remains practical as the number of training trajectories grows.
Thank you again for your thoughtful feedback. We hope the additional results and clarifications address your concerns.
Official Comment I
We thank the reviewer for the thoughtful review and for recognizing that SCRIPT generalizes well to unseen tasks and uses far fewer tokens than the baselines. Because the scalability analysis in W1 builds on how the router is learned, we first answer Q4 and Q5 on router generation and execution, and then address W1 and the remaining questions.
We have revised Section 3.3 to clarify how SCRIPT learns the router. We first clarify that our aim is not to develop a new program search procedure. Our motivation is to formulate skill selection as a coding task with automated tests and feedback, allowing us to leverage the problem-solving capabilities of coding agents. Population-based methods such as AlphaEvolve rely on extensive LLM sampling, in part because their evaluators typically return aggregate scores for each candidate program rather than identifying which part of the program is wrong. SCRIPT’s cached verifier reports every routing error, including the predicted skill, the expected skill, and the source line that produced the prediction. This feedback helps the coding agent identify which parts of the router to inspect and revise. The agent can thus refine a single program through successive local edits, allowing SCRIPT to benefit directly from advances in coding agents. This formulation puts skill selection in a form that coding agents already handle well. The same ability to autonomously write, test, and revise code allows them tackle complex tasks, from fixing software issues (SWE-agent) to modeling interactive environments (ARC-AGI-3). We describe the program generation process below.
Training data. Every skill boundary in a trajectory yields one decision point
(task, history) → skill, shown aspoints[i]in the diagram, and these points are the only data the coding agent receives. The objective is to write a functionchoose(task, history)that maximizes routing accuracy over all points with general and compact code, measured by the MDL score. A verifier runschooseon decision points and returns the MDL score, the accuracy, and each error with its gold skill, predicted skill, and source line, with no LLM call and no environment interaction. Together these form a complete workspace for an ordinary coding task, in which the agent only needs to focus on the conditions under which each skill applies and how skills are combined.Authoring loop. In every turn the agent can write arbitrary Python code (regular expressions, frequency counts, and helper functions) or call workspace tools. It typically spends two to four turns inspecting the most common skills and their contexts to infer the shared conditions, and then writes an initial router. After every revision, the verifier runs the router on all training decision points and reports the MDL score, the routing accuracy, and a list of errors (input context, the predicted skill, the expected skill, and the source line that produced the prediction). The agent uses this feedback to edit the responsible conditions and reruns the verifier. Because verification needs only Python execution, the agent can test both complete router revisions and individual conditions as often as it wants. Over the session, the router gradually captures more of the rules that govern how skills are ordered and combined.