acceptodds
Under review as a conference paper at ICLR 2027

RSI-Router: Cost-Efficient LLM Routing via Evolution of Subtask-Level Model Assignments with Execution Skills

40 views

Abstract

Practical deployment of large language model (LLM) agents requires strong task performance at affordable inference cost. For long-horizon agentic tasks, this performance–cost trade-off can be improved through within-task large–small model collaboration, as smaller models can handle some stages even when they cannot solve the full task. In this paper, we introduce RSI-router, a routing framework that constructs subtask-level model assignments and model-specific execution skills through recursive self-improvement over execution experience. Each iteration consists of four stages: Subtask Mining derives subtask definitions and identification rules from training trajectories; Routing Strategy Evolution proposes and evaluates diverse model assignments; Model-Specific Skill Evolution compares routed and large-model-only trajectories to diagnose failures and develop reusable execution skills; and Pareto-Optimal Router Selection updates the Pareto population from historical and new routers while retaining dominated routers for reuse. Across five agentic benchmarks, RSI-router reduces inference cost by 51.7% on average and up to 82.2% while outperforming the large-model baseline. It also yields a stronger performance–cost Pareto frontier than existing routing baselines. Further analyses show that RSI-router progressively reduces cost over iterations without performance loss, building on previously accumulated skills.

Primary area:
foundation or frontier models, including LLMs
Keywords:
LLM routing, cost-efficient inference, self-evolving agents

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_ByGFOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

RSI-router jointly evolves subtask definitions, large/small model assignments, and model-specific execution skills. A proposal agent analyzes training trajectories, proposes candidate routers, improves their skills, and maintains a validation-based Pareto population. At inference time, the small model identifies the current subtask and selects its assigned executor. Across five benchmarks, the paper reports an average 51.7% online cost reduction while the selected configurations match or exceed the large-model baseline. The approach is promising, but the selection protocol, limited test samples, and incomplete attribution of the gains leave the claimed advance insufficiently established.

Critical strengths
  1. Jointly optimizing model assignment and execution guidance is well motivated: a skill can change what a small model can reliably execute, so routing against fixed capabilities can be unnecessarily restrictive. The training/validation separation for trajectory access and the use of a small model for runtime routing are sensible design choices.

  2. The paper evaluates several distinct agent environments and includes routing calls and skill context in online token costs. The large reductions on ALFWorld, ScienceWorld, and WebShop are encouraging. It also reports the unfavorable side of the skill ablation: skills can reduce coding-task success while saving cost. This is more informative than assuming that every skill necessarily improves accuracy.

Critical weaknesses
  1. The headline comparison appears to involve selection using test outcomes. Section 3.1 evaluates every final Pareto router on the test set, and Section 3.2 reports the cheapest strategy whose measured performance reaches the large-model baseline. Unless that final choice was fixed using validation data alone, the reported operating point is selected with test feedback, even if test trajectories never guide evolution. This is especially consequential with only 32 SWE-bench and 25 Terminal-Bench test tasks. The SWE-bench difference, 84.38% versus 83.33% over three repeats, corresponds to approximately one additional successful trial among 96. Table 1 lacks paired uncertainty for these selected performance/cost comparisons. Repeating inference on the same tasks does not address variability from task sampling and router evolution.

  2. The experiments do not adequately isolate the contribution of recursive subtask routing from execution-skill or harness optimization. SkillOrchestra, which already learns skill-conditioned routing from execution experience, is discussed but not evaluated. More generally, the paper lacks large-only and small-only controls receiving comparably evolved execution skills, and a budget-matched fixed-subtask search. Masking skills after joint optimization is useful but does not substitute for these comparisons; indeed, it raises SWE-bench and Terminal-Bench performance by 9.7 and 10.5 points while increasing cost (Figure 6(c)). Thus the evidence establishes a promising combined system more clearly than an advance attributable to the proposed routing/evolution mechanism.

Questions for authors
  1. Selection clarification: Were the exact Table 1 configurations fixed before observing any test results? Please report validation-selected operating points with task-paired uncertainty, and distinguish inference repeats from independent router-evolution runs.

  2. Contribution evidence: Is a comparison with SkillOrchestra, or with the same skill-evolution procedure applied to fixed large-only/small-only executors, available? Which gain remains when proposal/search compute and execution guidance are matched?

  3. Cost clarification: What are the offline proposal and candidate-evaluation costs, and the resulting deployment break-even workload? How sensitive is the online frontier to the assumed 1:30 small/large price ratio, given that Appendix C's six measured GPU-hour ratios range from 25.32 to 99.15?

5 comments
AuthorsOpenReview ↗

Response to Reviewer ByGF Part I

We appreciate your careful and constructive feedback! Below we provide our responses, and we are happy to follow up on any additional questions.


R-W1: Test-based selection and uncertainty

Several methods, including ours, produce multiple operating points. Table 1 therefore summarizes every method with the same reporting rule: the cheapest of its configurations that reaches large-only performance on the test tasks. To make the reported point independent of test results, we repeated the selection with the same rule on validation data only: for each benchmark, we take the cheapest router on the validation Pareto front whose validation performance is at least that of the large model, and evaluate it on the test set afterwards. The validation-selected routers still reduce cost by 48.9% on average (51.7% in Table 1) and match or exceed the large model on four of the five benchmarks.

The table reports these routers on the test sets of Table 1 (three repeats per task; ± is the half-width of a 95% interval from a task-level paired bootstrap):

BenchmarkTest tasksLarge-onlyRSI-router (validation-selected)Δ (± 95% CI)Cost saving (± 95% CI)
ALFWorld6490.698.4+7.8 ± 6.475.3% ± 7.2
ScienceWorld6434.437.0+2.6 ± 11.075.5% ± 6.8
WebShop6459.262.3+3.1 ± 5.367.0% ± 8.6
SWE-bench Verified3283.384.4+1.0 ± 4.58.1% ± 46.9
Terminal-Bench 2.02540.034.7−5.3 ± 7.218.8% ± 19.6

On SWE-bench, the rule picks the same router as Table 1; as you point out, its gap to the large model (84.38 vs. 83.33) is one trial out of 96, so we do not claim a performance gain there. On this benchmark, the router's benefit is its lower cost. Terminal-Bench is the exception: the rule picks a cheaper router that is 5.3 points below the large model at 18.8% lower cost. The revision will report the validation-selected routers for all benchmarks.

AuthorsOpenReview ↗

Response to Reviewer ByGF Part II

R-W2: Attribution of routing versus skills and harness optimization

We added the four comparisons you ask for. None of them reaches RSI-router's performance at a lower cost. Each cell is score / cost (USD over the test set):

MethodALFWorldScienceWorldWebShopSWE-bench VerifiedTerminal-Bench 2.0
Large-only90.6 / \$1.5034.4 / \$1.9159.2 / \$0.7583.3 / \$3.4640.0 / \$17.10
Small-only65.6 / \$0.1314.6 / \$0.0647.5 / \$0.0262.5 / \$0.695.3 / \$0.17
SkillOrchestra87.5 / \$0.7731.3 / \$1.2647.5 / \$0.4071.9 / \$3.2520.0 / \$6.47
RSI-router (validation-selected)98.4 / \$0.3737.0 / \$0.4762.3 / \$0.2584.4 / \$3.1834.7 / \$13.88

Comparison with SkillOrchestra. We use the official implementation with the same training trajectories as RSI-router. SkillOrchestra extracts its skills from pre-collected trajectories and writes them for the router, to decide which model executes a step; the executing model receives no guidance. RSI-router instead re-executes every candidate router and writes skills for the executing model. In the results, SkillOrchestra saves 6–62% of the large-only cost but loses 3–20 points on every benchmark. RSI-router scores 5.7–14.8 points higher than SkillOrchestra on all five benchmarks, at lower or equal cost on four. On Terminal-Bench, SkillOrchestra is cheaper (\6.47vs.6.47 vs. \\13.88), but its success rate is half that of the large model (20.0 vs. 40.0; RSI-router: 34.7).

The three controls are on ALFWorld, ScienceWorld, and WebShop, where the validation-selected router uses skills:

MethodALFWorldScienceWorldWebShop
Large-only90.6 / \$1.5034.4 / \$1.9159.2 / \$0.75
Large-only + evolved skills100.0 / \$1.0857.8 / \$2.4963.7 / \$1.09
Small-only + evolved skills75.0 / \$0.1321.9 / \$0.1257.1 / \$0.04
Fixed subtasks96.9 / \$0.5139.1 / \$0.7556.8 / \$0.12
RSI-router (validation-selected)98.4 / \$0.3737.0 / \$0.4762.3 / \$0.25

With evolved skills but no routing, neither model reaches RSI-router's trade-off. In these two controls, one model executes every step and gets its own six-iteration skill evolution with the same proposal agent. RSI-router costs 66–81% less than the large model with skills and scores 5–23 points higher than the small model with skills. The large model with skills is on par with RSI-router on ALFWorld and WebShop (within 1.6 points); on ScienceWorld it is 20.8 points higher but costs 5.3× as much. Skills do not make it cheap: on ScienceWorld and WebShop it costs more with skills than without. The low cost therefore requires routing.

Fixed-subtask search. We added this control: a new six-iteration search with the same number of candidates, in which the subtask definitions stay at their initial version and only model assignments and skills evolve. The router is selected with the validation rule of R-W1. With fixed subtasks, the search already saves more than 60% of the large-only cost. Evolving the subtasks as well lowers cost further, by more than a quarter on ALFWorld and more than a third on ScienceWorld, at similar performance (within 2.1 points).

Skill masking (Fig. 6(c)). The gain from masking skills that you cite is the other side of a cost trade: on SWE-bench and Terminal-Bench, skills give up about 10 points of success and save about a third of the large-only cost, and the Pareto population keeps routers both with and without skills. The validation-selected routers there carry no skills.

AuthorsOpenReview ↗

Response to Reviewer ByGF Part III

R-Q1: Selection protocol and task-paired uncertainty

The Table 1 configurations were not fixed before testing (see R-W1). The operating points fixed on validation alone are in R-W1, with task-paired 95% intervals: they reduce cost by 48.9% on average (51.7% in Table 1) and match or exceed the large model on four of the five benchmarks.

The three repeats are inference repeats of one evolution run per benchmark. We added a second, independent evolution with a different small model (Qwen3.5-4B), with the router selected by the same validation-only rule:

BenchmarkLarge-only4B small-onlyRSI-router (4B)Δ (± 95% CI)Cost saving (± 95% CI)
ALFWorld90.644.895.3+4.7 ± 6.667.1% ± 13.4
ScienceWorld34.49.431.8−2.6 ± 10.667.2% ± 6.9
WebShop59.251.558.7−0.5 ± 4.865.3% ± 8.4
SWE-bench Verified83.342.776.0−7.3 ± 9.053.1% ± 13.7
Terminal-Bench 2.040.01.322.7−17.3 ± 13.480.1% ± 9.9

On ALFWorld, ScienceWorld, and WebShop the 4B router is within −2.6 to +4.7 points of the large model at 65–67% lower cost. On SWE-bench it trades 7.3 points for 53% lower cost. The only large loss is on Terminal-Bench (17.3 points), because the 4B model is too weak for these tasks: alone it solves 1 of 75 runs.

R-Q2: SkillOrchestra and single-model controls with evolved skills

Yes; both are in R-W2: SkillOrchestra in the first table and the single-model controls with evolved skills in the second. The controls use the same proposal agent and six iterations, and each fixed executor gets skills evolved for itself. What remains is the cost: RSI-router costs 66–81% less than large-only + skills, at comparable performance on two of the three benchmarks.

R-Q3: Offline cost and price sensitivity

Offline cost. One complete evolution run per benchmark (6 iterations × 4 candidates, each evaluated before and after its skill update, two repeats on training and validation):

ALFWorldScienceWorldWebShopSWE-benchTerminal-Bench
Candidate evaluation$154$276$66$965$1,103
Single-model trajectories$8$10$5$45$61
Proposal agent (DeepSeek-V4.1-Flash)$60$65$47$67$61
Total$222$351$118$1,077$1,225
Wall-clock13.6 h13.1 h6.5 h16 h41 h

This cost is paid once per benchmark. The evolved router is then used for every task, at 67.0–75.5% lower cost per task on ALFWorld, ScienceWorld, and WebShop, 8.1% on SWE-bench, and 18.8% on Terminal-Bench (R-W1).

Price ratio. Repricing every call of the same routers gives the following cost savings relative to large-only:

Small:large price ratio1:25.321:301:35.961:501:99.15
ALFWorld74.7%75.3%75.9%76.6%77.6%
ScienceWorld74.6%75.5%76.3%77.5%78.9%
WebShop65.5%67.0%68.4%70.3%72.7%
SWE-bench Verified7.9%8.1%8.3%8.5%8.9%
Terminal-Bench 2.018.7%18.8%19.0%19.2%19.4%

The small model accounts for only 1–25% of RSI-router's cost, so the savings move by −1.5 to +5.7 points over the measured range (25.32–99.15). The assumed 1:30 is also below the aggregate measured ratio (35.96), so the reported savings are conservative. At no ratio in this range does any method in Table 1 reach RSI-router's performance at a lower cost.

Reviewer_Jeh1OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes RSI-router, a routing framework that constructs subtask-level model assignments and model-specific execution skills through recursive self-improvement over execution experience. Each iteration contains subtask mining, routing strategy evolution, model-specific skill evolution, and pareto-optimal router selection. Experiments cover ALFWorld, ScienceWorld, WebShop, SWE-bench Verified, and Terminal-Bench 2.0. The paper reports an average test-time inference-cost reduction of 51.7%, with up to 82.2% reduction while matching or exceeding the large-model baseline.

Critical strengths
  1. The problem is well-motivated. Agentic tasks often contain stages with very different capability requirements, so whole-task routing can miss substantial savings.

  2. The overall formulation is coherent. Subtask mining, routing strategy evolution, model-specific skill evolution, and pareto-optimal router selection each address a recognizable part of the routing problem.

  3. The evaluation spans five distinct benchmarks and reports positive performance compared to existing baselines.

Critical weaknesses
  1. The paper claims to contribute “a recursive self-improvement paradigm for LLM routing.” However, its evolutionary process relies on a powerful proposal agent (Codex), and the system is closer to LLM-guided evolutionary search over routing assignments and prompt-like skills, with the proposal agent reading execution outcomes and proposing the next population.

  2. The computational overhead of the proposal agent remains unclear. The four-stage evolutionary loop requires a strong LLM (Codex) as the proposal agent to drive subtask mining, strategy generation, skill generation, and router selection. However, the paper reports only inference-stage costs and does not fully detail the computational cost of evolution itself, including proposal-agent calls, token consumption, and execution costs for trajectory collection. For a framework requiring six iterations with four candidate routing strategies per iteration, such costs may affect its practical utility.

  3. The small-model cost estimation appears somewhat oversimplified. The paper estimates the price of Qwen3.5-9B using a fixed 1:30 reference ratio informed by GPU-hour measurements, yet Table 2 shows that the actual ratio varies widely across input lengths (25.32–99.15). A fixed ratio may therefore introduce substantial pricing bias, particularly when input lengths vary considerably across tasks. A sensitivity analysis is needed.

  4. Only a single large–small model pair was evaluated. The experiments use DeepSeek-V4.1-Flash as the large model and Qwen3.5-9B as the small model. In the absence of experiments across different model pairs, it remains unclear whether the method’s effectiveness is sensitive to model choice. This raises an important concern regarding its generalizability.

Questions for authors
  1. What is the total computational cost of the evolutionary process? Please consider report the full evolutionary cost, including the total token consumption and number of calls for the proposal agent (Codex), the cost of trajectory collection, and the total time required to complete six rounds of iteration on each benchmark. This is essential for assessing the practical feasibility of the method.

  2. How accurate is subtask identification? Please consider report the accuracy of subtask identification and the extent to which identification errors affect final task performance.

  3. What is the rationale for selecting G = 6 iterations and N = 4 candidates? How do these two hyperparameters affect the final results? Is there a reasonable early-stopping criterion?

  4. How sensitive are the comparison results to the assumed small-model price ratio?

4 comments
AuthorsOpenReview ↗

Response to Reviewer Jeh1 Part I

We appreciate your careful and constructive feedback! Below we provide our responses, and we are happy to follow up on any additional questions.


R-W1: Recursive self-improvement versus LLM-guided evolutionary search

The proposal agent is the system's own large model, not a more powerful one. It is DeepSeek-V4.1-Flash, the same model as the large model L. Codex CLI is only the harness in which this model reads the execution trajectories and writes its proposals.

Following recent work (Meta-Harness; Self-Harness; RRSI), we use "recursive self-improvement" for a system that keeps its model weights fixed and improves what surrounds the model from its own runs, with each improved version generating the feedback for the next.

  • Self. The model that executes the tasks also reads the system's own execution trajectories and rewrites its subtask definitions, model assignments, and skills. Nothing in a router is written by hand, and no stronger model is involved.
  • Recursive. Each iteration starts from the routers, subtasks, and skills of the previous one. The cost at fixed performance falls progressively over the iterations (Fig. 4), and skills written in early iterations are reused in later ones (Fig. 5).

The procedure is LLM-guided evolutionary search. The two descriptions do not conflict: one names the procedure, the other names what improves and by whom.

References

  • Lee, Y., Nair, R., Zhang, Q., Lee, K., Khattab, O., & Finn, C. (2026). Meta-harness: End-to-end optimization of model harnesses. arXiv preprint arXiv:2603.28052.
  • Zhang, H., Zhang, S., Li, K., Zhang, C., Chen, Y., Zhang, Y., ... & Hu, S. (2026). Self-harness: Harnesses that improve themselves. arXiv preprint arXiv:2606.09498.
  • Xia, P., Han, R., Wang, Z., Chen, Y., Zhuang, Y., Lee, Y., ... & Lee, C. Y. (2026). RRSI: Regularized recursive self-improvement of agent harnesses. arXiv preprint arXiv:2609.24972.

R-W2: Offline proposal and evolution costs

The proposal agent costs \$47–67 per benchmark; the rest of the offline cost is collecting execution trajectories, and the whole cost is paid once. One complete evolution per benchmark (6 iterations × 4 candidates, each evaluated before and after its skill update, two runs per training and validation task, prices of Table 1):

ALFWorldScienceWorldWebShopSWE-benchTerminal-Bench
Candidate evaluation\$154\$276\$66\$965\$1,103
Single-model trajectories\$8\$10\$5\$45\$61
Proposal agent (DeepSeek-V4.1-Flash)\$60\$65\$47\$67\$61
Wall-clock13.6 h13.1 h6.5 h16 h41 h

For agentic tasks, this execution cost is hard to avoid. When a router switches models between subtasks, the rest of the episode changes. Its performance therefore cannot be computed from the trajectories of the two single models, as it can for single-turn queries; every candidate has to be run. Each run is an episode of tens of steps with a growing context, so the cost follows the price of an episode: \0.005perrunonWebShopand0.005 per run on WebShop and \\0.18 on Terminal-Bench.

The proposal agent makes 1,618–2,460 model calls and uses 146–212M input tokens and 2.5–3.5M output tokens per benchmark. Once evolved, the router serves every later task, so this one-time cost is spread over its whole deployment. Without the search, the subtasks, model assignments, and skills would have to be designed by engineers reading trajectories by hand.

R-W3: Small-model pricing assumptions and sensitivity

The fixed ratio does not favor us: 1:30 is below the aggregate measured ratio (35.96), and a higher ratio only increases RSI-router's savings. We report routers selected on validation data only (the cheapest router whose validation performance is at least that of the large model). Repricing every call of these routers on the test sets of Table 1 gives the following cost savings relative to large-only:

Small:large price ratio1:25.321:301:35.961:501:99.15
ALFWorld74.7%75.3%75.9%76.6%77.6%
ScienceWorld74.6%75.5%76.3%77.5%78.9%
WebShop65.5%67.0%68.4%70.3%72.7%
SWE-bench Verified7.9%8.1%8.3%8.5%8.9%
Terminal-Bench 2.018.7%18.8%19.0%19.2%19.4%

The savings are stable because the small model accounts for only 1–25% of RSI-router's cost: they move by −1.5 to +5.7 points over the whole range. At no ratio in this range does any method in Table 1 reach RSI-router's performance at a lower cost.

AuthorsOpenReview ↗

Response to Reviewer Jeh1 Part II

R-W4: Generalization across model pairs

We added a second, complete evolution on all five benchmarks with the small model replaced by the smaller Qwen3.5-4B, and selected the router on validation data only, with the same rule as in R-W3:

BenchmarkLarge-only4B small-onlyRSI-router (4B)ΔCost saving
ALFWorld90.644.895.3+4.767.1%
ScienceWorld34.49.431.8−2.667.2%
WebShop59.251.558.7−0.565.3%
SWE-bench Verified83.342.776.0−7.353.1%
Terminal-Bench 2.040.01.322.7−17.380.1%

On ALFWorld, ScienceWorld, and WebShop the 4B router is within −2.6 to +4.7 points of the large model at 65–67% lower cost. On SWE-bench it trades 7.3 points for 53% lower cost. The only large loss is on Terminal-Bench (17.3 points), because the 4B model is too weak for these tasks: alone it solves 1 of 75 runs.

R-Q1: Total evolution cost and wall-clock time

See R-W2 above: cost, proposal-agent usage, and wall-clock time for the six iterations on each benchmark.

R-Q2: Subtask identification accuracy and downstream impact

The evolution defines the subtasks itself, so identification accuracy has no external ground truth.

The effect of identification errors on task performance, however, is already included: every reported result uses online identification by the small model, errors included.

Identification is also optimized during evolution. Subtask Mining replays training trajectories, compares the small model's predictions with the proposal agent's labels, and rewrites ambiguous rules; a router whose subtasks are hard to identify does worse on validation and is not selected. Giving the identifier a longer history does not improve task performance either (Fig. 6(a)).

R-Q3: Iteration count, population size, and early stopping

G and N set the search budget. We used one setting for all benchmarks and did not tune it, since every setting needs a complete evolution per benchmark.

G. At fixed performance, the validation cost falls over the iterations and last improves in iterations 3–5 (Fig. 4): after three iterations it is already 19.5–78.2% below the large model, and 34.5–78.2% after six.

N. For the same total budget, a larger N tries more ideas in each iteration, and a smaller N leaves more iterations to build on the best ones.

Early stopping. Gains arrive irregularly (SWE-bench's last drop follows two stalled iterations), so a rule needs patience: stop after three iterations without improvement.

R-Q4: Sensitivity to the small-model price ratio

See R-W3 above. The range 25.32–99.15 is the six GPU-hour ratios measured in Appendix C for the two models (552B vs. 9B parameters), and 1:30 is below their aggregate (35.96). Over this range, the savings move by only −1.5 to +5.7 points, and no method in Table 1 reaches RSI-router's performance at a lower cost at any of these ratios.

We sincerely thank you for your valuable feedback. We hope our responses have been helpful in addressing your concerns. We would be grateful to know whether these clarifications have resolved the points you raised. Please do not hesitate to let us know if any questions remain, we are happy to provide further discussion.

Thank you for your thoughtful feedback. As the discussion period draws to a close, are there any further questions or concerns you would like us to address? Could you let us know whether our rebuttal addresses your concerns? If so, please consider raising your score. We would be happy to clarify any remaining points.

Reviewer_wmrgOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes RSI-router, a training-free routing framework that divides agentic tasks between a large and a small model to reduce cost. Offline, Codex iteratively segments trajectories into subtasks, assigns a model to each subtask, writes model-specific skills (prompt text), and keeps Pareto-optimal routers based on validation performance and cost. At inference time, the small model identifies the current subtask, selects the assigned model, and appends the matching skills. Across five agentic benchmarks, the paper reports matching or exceeding large-model-only performance at 51.7% lower cost on average, outperforming existing routing baselines. However, the validation remains at the level of aggregate numbers: the paper does not show what problems were discovered on each benchmark or how skills are organized, added, and removed, and the large-model baseline is weak while the test sets are small.

Critical strengths

S1. The method consistently outperforms random mixing at matched cost. Using the numbers in Table 1, I computed the expected performance of randomly assigning each task to the large or small model at the same cost. RSI-router exceeds this line on all five benchmarks (+28.4 on ALFWorld, +18.4 on ScienceWorld, +10.4 on WebShop, +3.2 on SWE-bench Verified, and +13.0 on Terminal-Bench 2.0), whereas all baselines stay within about ±7 points of it. I consider this the most convincing evidence in the paper that subtask-level collaboration provides benefits beyond what task-level routing can achieve, rather than merely trading off between the two models.

S2. The evolution process is tracked and the evaluation protocol is clear. The training, validation, and test sets are disjoint, the test set provides no feedback, and the reported cost includes routing calls and skill context. Figure 4 shows that cost decreases steadily across iterations under a fixed performance constraint, and Figure 5 shows that skills from early iterations continue to be reused later. This supports the claim that iterative evolution helps beyond the final numbers alone, and makes the method’s behavior more transparent than many self-evolving approaches.

Critical weaknesses

W1. The paper does not show what the evolution process actually discovers, and the skill mechanism is not analyzed. The central claim is that skills enable models, especially the small model, to accomplish what they could not before. However, for each benchmark the paper only provides one-line skill summaries in Appendix D. It does not report which problems were discovered, which previously failing cases succeed because of the skills, how long the skills are, how they are organized by problem type, or how the method decides to add or remove a skill. These matter: adding skills too quickly leads to so many skills that the model may ignore them, adding too slowly may miss useful ones, and good skills should address many problems, whereas most examples target a single task type (e.g., “use the lamp” in ALFWorld). The available evidence even points the other way: removing skills improves performance by about 10 points on SWE-bench Verified and Terminal-Bench 2.0, where the skills mostly restrict retries, suggesting that they trade performance for cost rather than extend capability. As a result, I cannot confirm that the core mechanism works as claimed.

W2. The gains may come from benchmark-specific prompts written by a strong model rather than from routing. The absolute performance of the large-model baseline is low; for example, its WebShop reward of 0.59 is roughly the level of Qwen2.5-7B with Reflexion, indicating a minimal harness. RSI-router exceeds the large model on several benchmarks, but routing alone cannot exceed the large model’s performance, so this part of the gain must come from the skills written by Codex. Yet the key baseline of the large model with the evolved skills is missing. In addition, the test sets contain only 25 to 64 tasks, the +16.7% gain on Terminal-Bench 2.0 corresponds to only 1.67 tasks, and the reported router may have been selected on the test set. Under these conditions, the main results do not support the conclusion that subtask-level routing itself is effective.

Questions for authors

Q1. For each benchmark, what problems did the evolution process discover automatically? Can the authors report the failure modes identified during evolution, their frequency before and after the corresponding skills were added, and examples of tasks that changed from failure to success because of a skill? Can they release the full final skill library?

Q2. How does the method decide when to add, merge, or remove a skill? Is there any per-skill validation before a skill enters the shared library, and how large is the library after each iteration?

Q3. How long are the skills in tokens, how does the per-step context (skills and subtask identification rules) grow across iterations, and what fraction of total test cost comes from skill context and routing calls?

Q4. Are the diagnosed failures organized by problem type? For each skill in the final router, how many failure categories and what fraction of tasks does it cover?

Q5. Which split were the ALFWorld and WebShop test tasks drawn from, how were the 64 ScienceWorld tasks selected across task types, and what step limit was used for each benchmark? Can the authors report both average score and success rate for ScienceWorld and WebShop?

Q6. How does the large model perform when given the evolved skills without any routing? If it matches RSI-router’s performance at a similar or lower cost, what is the remaining contribution of routing?

Q7. Was the router reported in Table 1 selected using validation or test results? If selected on test, how do the results change when a single router is chosen on validation?

6 comments
AuthorsOpenReview ↗

Response to Reviewer wmrg Part I

We appreciate your careful and constructive feedback! Below we provide our responses, and we are happy to follow up on any additional questions.

R-W1: Evidence for discovered failure modes and the skill mechanism

Skills act differently on the two kinds of benchmarks. On the three interactive benchmarks (ALFWorld, ScienceWorld, WebShop), they fix recurring failures and raise performance; on the two coding benchmarks (SWE-bench Verified, Terminal-Bench 2.0), they remove wasted steps and lower cost. All numbers are on the test sets of Table 1.

Skills add 7–12 points on the three interactive benchmarks. We added cost measurements to the skill-removal experiment of Fig. 6(c), which re-runs all 34 routers that carry skills with their skills removed:

Skills removed → with skillsALFWorldScienceWorldWebShopSWE-benchTerminal-Bench
Performance85.1 → 97.526.4 → 37.954.5 → 61.980.6 → 70.832.0 → 21.5
Cost (% of large-only)95 → 2938 → 4731 → 3090 → 5666 → 32

On the interactive benchmarks, the small model alone also gains 7–10 points from its own evolved skills (65.6 → 75.0, 14.6 → 21.9, 47.5 → 57.1).

Each skill is a general rule against one recurring failure. We analyzed the execution trajectories and added a case study for each benchmark. In the first three rows the skill raises success; in the last two it saves steps:

BenchmarkWhat goes wrong without skillsRule the skill gives (paraphrased)Skills removed → with skills
ALFWorld (household tasks)Does not notice that the task is unfinished. After a mistake, such as putting away the wrong object, the agent keeps checking its surroundings until its steps run out.If the task is not complete, compare what you did with the goal and do the missing step.Runs stuck this way: 30 → 0
ScienceWorld (science experiments)Answers with the wrong thing. Asked to select a plant or a liquid, the agent selects the pot or cup that holds it.Select exactly what the task names, not its container.Runs ending on this wrong answer: 40 → 15
WebShop (online shopping)Repeats an action that did not work. The agent submits the same search again and gets the same results.A repeated search returns the same results; change the query or open an item not yet tried.Searches that repeat an earlier one: 65% → 35%
SWE-bench (fixing code)Tests far more than needed. The agent runs whole test suites where a few tests cover the change.Run only the tests for the code you changed.Steps to solve the same task (median): 21 → 15
Terminal-Bench (command-line tasks)Checks far more than needed. The agent re-reads files and re-runs its own checks many times before it hands in the result.Run a check once, keep its output, and act on it.Steps to solve the same task (median): 29 → 17

These rules are not memorized answers: the test rooms, the product categories, and 12 of the 25 science task types never occur in training or validation (on those 12 types, the ScienceWorld failure falls from 33 to 12 runs). R-Q1 gives test tasks that change from failure to success.

The library has many cards, but each step is given only the few that match its current subtask. Each card is one skill (a failure and a rule against it) of about 170 tokens and enters the library only with a cited failure. Length, organization, and the rules for adding and removing cards are in R-Q2–R-Q4.

On the coding benchmarks, skills cut cost at the price of about 10 points of success. Here your reading is correct. This is why the search keeps routers both with and without skills: the routers selected on validation data only (R-Q7) use none there, and they are on par with the large model on SWE-bench (84.4 vs. 83.3) and 5.3 points lower on Terminal-Bench. The saving from skills does not come only from solving fewer tasks either: when the same task is solved in both settings, the run with skills costs 34% less on SWE-bench and 50% less on Terminal-Bench. The revision will state the two kinds of effect separately.

AuthorsOpenReview ↗

Response to Reviewer wmrg Part II

R-W2: Skill-enhanced baselines and reliability of the evaluation

Routing is needed for the low cost: with its own evolved skills but no routing, the large model costs 2.9–5.3× as much as RSI-router. We added the baseline you ask for and two further controls (score / cost on the three interactive benchmarks):

ALFWorldScienceWorldWebShop
Large-only90.6 / \$1.5034.4 / \$1.9159.2 / \$0.75
Large-only + evolved skills100.0 / \$1.0857.8 / \$2.4963.7 / \$1.09
Small-only + evolved skills75.0 / \$0.1321.9 / \$0.1257.1 / \$0.04
RSI-router with skills removed84.4 / \$1.4826.6 / \$0.4653.6 / \$0.26
RSI-router (validation-selected)98.4 / \$0.3737.0 / \$0.4762.3 / \$0.25

Without its skills, RSI-router is 6–8 points below the large model. With evolved skills but no routing, the large model scores as high as RSI-router or higher (20.8 points higher on ScienceWorld) but costs 2.9–5.3× as much, and the small model is cheap but 5–23 points lower. Only the combination reaches the large model's performance at a quarter to a third of its cost. The points gained over the large model do come from the skills.

On the coding benchmarks, the validation-selected routers carry no skills, so their results come from routing alone. On SWE-bench the router is on par with the large model at 8% lower cost; on Terminal-Bench it is 5.3 points lower at 19% lower cost (R-Q7).

The skills are not written by a stronger model. The proposal agent is DeepSeek-V4.1-Flash, the same model as the large model L. Codex is only its harness. The submission did not state this; the revision will.

All methods share one harness, so the comparison is fair. The large-model scores are low only relative to other harnesses. All methods here share one minimal harness and the same step limits, so scores from other harnesses are not comparable; Reflexion, for example, uses multiple trials. The large model reaches 90.6% on ALFWorld and 83.3% on SWE-bench Verified.

Selected on validation only (R-Q7), RSI-router still reduces cost by 48.9% on average and matches or exceeds the large model on four of the five benchmarks.

AuthorsOpenReview ↗

Response to Reviewer wmrg Part III

R-Q1: Discovered failure modes, skill effects, and the final skill library

The problems the evolution discovered, the rules written for them, and their frequency with and without the skills are in the second table of R-W1, one per benchmark. Each card of the skill libraries names one such failure and a rule against it (80–129 cards per benchmark). We will release the libraries with per-card provenance.

On the same test task, the failure disappears once the skill is present. Three tasks that fail without skills and succeed with them:

BenchmarkTest taskWithout skillsWith skills
ALFWorldPut the pepper shaker in a drawerPuts away a salt shaker by mistake, then keeps checking the drawer until its steps run out (all three runs)Notices the mistake, fetches the pepper shaker, and succeeds (all three runs)
ScienceWorldPick the shortest-lived animalPicks the wrong animal (all three runs)Compares the animals and picks the right one (all three runs)
WebShopBuy white engineered-wood furniture under \$270Repeats the same search and never buys (two of three runs)Buys the right product within 6 steps (all three runs)

On SWE-bench and Terminal-Bench, skills do not add successes; they shorten runs that succeed either way (R-W1).

R-Q2: Skill addition, merging, removal, and library growth

A card is added only with evidence of a failure, merged if it repeats another card, and removed if later runs contradict it.

  • Add. After a router is run on the training tasks, the proposal agent compares its runs with the large model's runs on the same tasks and writes a card for each kind of failure it finds. A card is accepted only if it points to a concrete run that shows this failure.
  • Merge. A new card that restates an existing card for the same subtask and model is merged into it.
  • Remove. The proposal agent retires cards that later runs contradict. When a subtask is split or merged, its cards are reassigned or dropped.

Each card is checked for evidence; performance is validated for a router's whole set of cards. After an update, the router is run again with its new cards, and the update is kept only if, on validation, no other router is both better and cheaper. Validating each card separately would multiply the evaluation cost by the number of cards.

The library grows to 80–129 cards, but after selection by the proposal agent, a router typically carries 5–10 of them. Library size after each iteration:

Iteration123456
ALFWorld2344607991107
ScienceWorld27395776102129
WebShop192434466080
SWE-bench29486285104119
Terminal-Bench2041506580103

R-Q3: Skill length, context growth, and routing overhead

Cards have a median length of 158–180 tokens and a maximum of 263.

A step is given only the cards for its current subtask, so its skill text stays at a few hundred tokens, however large the library gets; the subtask identification rules grow, but only the small model reads them.

Iteration 1 → 6ALFWorldScienceWorldWebShopSWE-benchTerminal-Bench
Skill text per subtask (average tokens)117 → 161160 → 16591 → 136210 → 185107 → 228
Subtask identification rules (tokens)1,075 → 1,8241,228 → 2,8791,130 → 2,330963 → 1,6921,037 → 1,244

On average, the skill text per subtask stays between 91 and 336 tokens in every iteration, while the library grows 4–5× (R-Q2).

Routing calls and skill text together account for 0.5–21% of the test cost, and both are included in every reported cost.

ALFWorldScienceWorldWebShopSWE-benchTerminal-Bench
Routing calls5.9%9.7%15.8%0.9%0.5%
Skill text4.6%2.4%5.2%00

R-Q4: Failure categories and skill coverage

Cards are organized by subtask and executing model, and each card addresses one failure. For each card that the validation-selected router uses on the test set, the table gives the share of the 64 test tasks in which it is used (one number per card):

Cards usedShare of test tasks in which each card is used
ALFWorld8100%, 100%, 98%, 92%, 92%, 83%, 73%, 66%
ScienceWorld798%, 98%, 98%, 98%, 70%, 31%, 31%
WebShop9100%, 100%, 89%, 44%, 33%, 31%, 25%, 23%, 22%

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.