acceptodds
Under review as a conference paper at ICLR 2027

Self-Evolving Agents via Likelihood-Guided Tool-Space Optimization

Abstract

Self-evolving agents can continually improve their behavior, while tools define the executable action space through which they interact with the environment. However, exposing the full tool library to model introduces substantial irrelevant context and can impair tool-use decisions. We study tool-space self-evolution, where each recurring task type maintains a persistent tool space which is constructed from accumulated output experience. We identify three limitations of existing methods: (1) output-unaware selection: they rely primarily on tool descriptions or model priors rather than observed tool outputs; (2) statelessness across request: they select tools independently for each request without consolidating prior output experience into persistent task-specific state; (3) inference cost: they repeatedly search, rank, or reason over candidate tools for subsequent requests of the same task. We address these limitations through output-aware tool scoring, persistent task-specific tool spaces, amortized tool selection, and reusable configurations across models. We introduce LOTS (Likelihood-Only Tool Scoring), which evolves an agent's tool space from accumulated output experience while keeping model parameters fixed. After each request, LOTS holds the model's generated answer and estimates each tool's contribution by measuring how much the answer likelihood changes when its observed output is removed. These contributions are aggregated within each recurring task to rank tools and update its persistent space. Across three benchmarks, LOTS improves task performance while substantially reducing tool context. More importantly, sequential experiments demonstrate that task-specific spaces persist and continue to improve over time, while cross-model experiments show that learned configurations transfer across different models.

Primary area:
foundation or frontier models, including LLMs
Keywords:
self-evolving agents, tool-using language models, tool-space evolution

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_D9rMOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes LOTS, a method for constructing and continuously refining a task-specific tool space for tasks an LLM agent performs repeatedly. Existing tool selection methods often rely on tool descriptions or the model’s pre-existing knowledge; they often fail to fully leverage the actual outcomes of tool execution and require repeating the selection process for every new request.

LOTS keeps the model’s final generated response fixed and measures the change in the response’s log-likelihood when the observed results from a specific tool are removed. By aggregating these "leave-one-out" attribution scores across multiple requests belonging to the same task, the method identifies high-scoring tools to form a persistent, task-specific tool space. When subsequent requests of the same type arise, the method can reuse this existing tool space and further update it with results from new executions.

Critical strengths

1. Efficient Tool Selection Leveraging Actual Tool Outputs : The primary strength of this paper lies in its approach: rather than relying solely on tool descriptions or semantic relevance, it measures the contribution of actual tool outputs, observed during execution, to the model’s final response. Specifically, the method of holding the final response constant and observing changes in likelihood when evidence from a given tool is removed is highly intuitive; furthermore, it can be applied without the need for additional model training.

2. Consistent Experimental Results Across Diverse Settings : The authors evaluate their method across three benchmarks: compositional tool use, multi-turn function calling, and executable ReAct agents. In the TGB benchmark, the method achieved higher accuracy across all eight tested models compared to using the full tool library; it also demonstrated positive results in the BFCL and GTA-Atomic benchmarks.

Critical weaknesses

1. Discrepancy between the claim of "label-free tool optimization" and the actual evaluation setup: A core feature of LOTS is that it estimates the importance of tools based on the model’s own outputs, without requiring ground-truth labels. However, the paper notes that the final tool budget, KK, is selected based on accuracy on a "fitting set." In other words, while the likelihood-based ranking itself does not utilize correctness labels, such labels are indeed used during the construction of the final tool space. Although the paper acknowledges this distinction and provides comparisons under a fixed budget, it remains unclear whether the primary experimental results can be replicated in a truly label-free deployment scenario. Specifically, further validation is required to determine the extent to which the task-specific selection of KK contributes to overall performance, and whether similar performance can be maintained if KK is determined without access to correctness labels.

2. Insufficient validation of "continual self-evolution" and actual task utility The paper posits that agents can continuously improve through persistent, task-specific tool spaces. Yet, in the primary sequential experiments, the task families or API classes are pre-defined, with tool spaces constructed and reused for each specific task. While these results demonstrate the utility of task-specific tool configurations, they do not sufficiently verify whether an agent can reliably adapt to real-world environments where task requirements shift or task boundaries are ill-defined.

Although the appendix includes an analysis of unsupervised task clustering, these experiments rely on synthetic task families with relatively clear-cut distinctions. Furthermore, likelihood-based attribution measures only the model’s reliance on its own generated responses—not the correctness of those responses or their actual utility. Because a model might assign a high score to a specific tool even when its output is incorrect, we need to examine more directly how tool-space quality evolves as errors accumulate over multiple updates. While the current experiment effectively demonstrates the benefits of tool-space optimization, it has limitations in proving the feasibility of reliable self-evolution in more general environments.

Questions for authors

Q1. Label-free tool selection: While LOTS uses likelihood-based scoring without correctness labels, the tool budget K is selected using fitting accuracy. I appreciate the fixed-budget results in Appendix, but how would K be determined when ground-truth answers are unavailable? Could the authors evaluate a fully label-free selection strategy to clarify how much the choice of K contributes to the reported gains?

Q2. Adaptation to changing tool requirements: The sequential experiments show that LOTS can refine tool spaces over time, but the task requirements remain largely stable. How would LOTS adapt if an existing task started requiring previously pruned tools? Figure 4 shows some recovery of missing tools, but it is unclear how reliably this works when tool requirements change during deployment.

Q3. Likelihood contribution versus task correctness: A tool that increases the likelihood of a generated answer may not necessarily improve its correctness. Although Appendix E.5. examines incorrect outputs, it is unclear whether these findings hold under repeated tool-space updates. Could incorrect outputs accumulate misleading attribution scores and eventually reinforce suboptimal tool selections?

Q4. Sensitivity to task grouping LOTS assumes that requests can be grouped into recurring tasks. Appendix D.1 shows that merging task families can hurt performance, particularly with smaller tool budgets. How robust is LOTS to imperfect task grouping, especially when semantically similar requests require different tools? Can the method recover from incorrect task assignments over time?

3 comments
AuthorsOpenReview ↗

Response to Reviewer D9rM

Thank you for recognizing the value of scoring actual tool outputs. We clarify the distinction between label-free scoring and budget selection, and report additional analyses of changing requirements, incorrect histories, and task grouping.

----- W1 / Q1. Is the complete tool-selection procedure label-free when KK is selected using fitting accuracy? -----

The likelihood score is label-free, while the submitted accuracy-based budget selection uses correctness labels. Fully label-free deployment can use a fixed budget, with a measurable performance trade-off. These properties should be reported separately.

We evaluated budget policies on TGB across eight models. The full accuracy curve over K∈{1,…,15}K\in\{1,\ldots,15\} is used to evaluate the selected budgets; only the explicitly labeled oracle uses test accuracy to choose KK. For off-model calibrated rules, the global constant is chosen on seven models and evaluated on the remaining model. Avoiding target-model tuning does not by itself establish that the entire calibration procedure is supervision-free, so we distinguish these results from fixed-budget deployment.

Budget policyBudget selection informationMean success (%)Gain over Keep all (pp)
Oracle budget, diagnostic upper boundTest labels79.84+14.34
Submitted task-specific K⋆K^\star80 labeled fitting requests per task79.30+13.80
Task-specific K⋆K^\star with two labelsTwo labeled fitting requests per task74.61+9.11
Mass-coverage rule, off-model calibrationConstant selected on the other seven models73.39+7.89
Fixed K=3K=3No labels required to apply the fixed budget71.50+6.00
Mean-contribution threshold, off-model calibrationConstant selected on the other seven models71.36+5.86
Keep allNone65.500

Fixed K=3K=3 retains a six-point gain over Keep all. Two labeled requests per task raise the average to 74.61%, compared with 79.30% under the submitted 80-request budget selection. The mass-coverage rule reaches 73.39% without target-model calibration, but its off-model tuning should not be described as globally supervision-free. These results quantify the contribution of budget selection.

We will incorporate the two-label and fixed-budget results as additional operating points, reserve “label-free” for score estimation and label-independent configuration choices, and retain the correct labeling of the submitted budget protocol.


----- W2 / Q2. Can LOTS recover when an existing task begins to require previously pruned tools? -----

Yes, with exploration enabled. In our task-shift experiment, LOTS recovered the previously pruned required tools in all 27 evaluated cases, including 25 within 40 requests.**

We tested precisely this scenario across five TGB models: the task identity remains fixed, but its required tool set changes at round 4, making previously pruned tools necessary. LOTS revises the existing task configuration using subsequent execution evidence. Mean post-change success reaches 68.2% at ε=0.1\varepsilon=0.1 and 70.6% at ε=0.25\varepsilon=0.25; recovery is rare without exploration. Recovery is possible because pruning excludes a tool from the current serving space, but does not delete it from the registry. Exploration re-exposes tools outside that space. Their newly observed outputs update the running likelihood scores, allowing previously pruned tools to re-enter the top-KK configuration. Thus, the configuration is revisable rather than permanently restricted to the tools retained during initial fitting.

We will add this experiment and clarify the recovery condition: LOTS can recover when exploration supplies fresh evidence about newly required tools, as demonstrated under the evaluated abrupt shifts.

AuthorsOpenReview ↗

Part 2/2 — Response to Reviewer D9rM

----- W2 / Q3. Can incorrect outputs accumulate misleading scores and reinforce poor selections? -----

This is a valid failure mode. A tool can receive a positive likelihood score for supporting an incorrect answer, so the score alone cannot prevent misleading evidence from influencing later selections. We tested whether incorrect fitting answers produce poor tool spaces and whether selected spaces are more vulnerable to erroneous tool outputs. These experiments address parts of the concern, but do not establish resistance to error amplification over repeated updates.

First, we constructed tool rankings separately from traces with correct and incorrect final answers, then evaluated the selected spaces at the same task-specific budget. Correctness labels are used only to form these diagnostic subsets; they are not inputs to the likelihood score. This tests whether an incorrect final answer necessarily makes its execution history unsuitable for tool selection.

ModelCorrect fitting answersLOTS, all traces (%)Correct-only ranking (%)Wrong-only ranking (%)
Qwen2.5-7B575 / 80082.4182.7580.72
Llama-3.1-8B609 / 80085.7586.0985.16
Mistral-7B360 / 80065.8165.9162.53
Qwen3-4B395 / 80068.3462.8868.03
Qwen3-8B476 / 80082.2880.3470.88

The wrong-only rankings still select useful spaces, but the effect varies by model. For example, Llama-3.1-8B achieves 85.16% with the wrong-only ranking versus 85.75% with all traces, whereas Qwen3-8B drops from 82.28% to 70.88%. Across 100 model–family–subset cases, including both correct-only and wrong-only subsets, 79 are within one percentage point of the all-trace result. Nine lose more than five points: four correct-only and five wrong-only. Thus, incorrect answers do not invariably destroy the selection signal, but the Qwen3-8B result shows that selection can deteriorate substantially.

Second, we tested erroneous tool outputs while keeping the selected spaces fixed. In an eight-model TGB experiment, a required tool returns an incorrect value on corrupted requests. Among requests that both LOTS and Keep all answer correctly under clean conditions, success under corruption is 17.17% for LOTS and 16.64% for Keep all. Both perform poorly under corruption, with similar aggregate success rates. This comparison tests sensitivity during execution; because the spaces are fixed, it does not test whether corrupted evidence leads to progressively worse selections.

The remaining question is whether errors reinforce themselves when new scores change the tool space and that space determines later execution histories. Neither diagnostic follows this feedback loop over repeated updates. We therefore treat long-term error amplification as an unresolved limitation, rather than infer stability from the fixed-history or fixed-space results.

We will incorporate these diagnostics and their limitations into the paper, clarify that likelihood attribution measures reliance on evidence rather than correctness, and explicitly state that resistance to error amplification over repeated updates has not been established.


----- W2 / Q4. How sensitive is LOTS to imperfect task grouping, and can it recover from incorrect assignments? -----

Grouping quality and tool budget jointly affect performance. In the tested misrouting experiment, the space recovers after routing is corrected externally; LOTS does not itself identify or repair the assignment error.

Text-only grouping matches oracle-grouping performance on the four-model TGB evaluation. Separate text-clustering diagnostics recover the evaluated TGB families and BFCL API classes.

Controlled merging into five groups reduces accuracy by up to five points at the fitted budgets, compared with 14.4–32.9 points at a fixed two-tool budget. A merged group can require more tools than the budget permits, making this both a grouping issue and a budget constraint.

For recovery, another family's requests are misrouted into a task's history during rounds 2–4, and correct routing resumes from round 5. Required tools are present within one corrected round in all 45 cells. In all 30 cells with exploration, accuracy from the second corrected round is within three points of the pre-misrouting level. Exploration also causes larger damage while misrouting continues, illustrating its sensitivity to the quality of incoming history.

We will incorporate the budget and recovery controls into the paper, state the dependence on supplied or inferred task groups, and distinguish recovery after external routing correction from autonomous correction of task identity.

Dear Reviewer D9rM,

As the author–reviewer discussion period draws to a close on Nov. 18, we would appreciate any further thoughts you may have on whether our response addresses your main concerns.

Thank you again for your time and consideration!

Reviewer_u4QAOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2(+1)
Presentation
3
Soundness
1
Read review
Paper summary

This paper introduces Likelihood-Only Tool Scoring (LOTS), a method for building persistent, task-specific tool spaces from an agent's past execution experience by measuring how much removing each tool's observed output changes the likelihood of the original response. These scores are aggregated across requests from the same task to rank and select useful tools. The selected tool space is then reused for future requests and can continue to evolve as more requests are observed, without updating the model parameters.

While LOTS shows improvements in task performance and efficiency across TGB, BFCL, and GTA-Atomic, the experimental evaluation has several critical weaknesses. LOTS benefits from in-distribution fitting requests and execution trajectories that are not equally used by several baselines, making it difficult to isolate the benefit of the proposed likelihood-based scoring. The evaluation also remains largely within the same task and tool distributions, leaving its robustness under distribution shifts unclear

Critical strengths
  • Simple and intuitive approach. LOTS uses a straightforward estimate for the importance of a tool from actual execution traces. The leave-one-out likelihood score is easy to understand and measures how much the model relies on information produced by each tool.
  • Persistent tool spaces. Instead of selecting tools from scratch for every request, LOTS learns a smaller tool space for recurring tasks and reuses it. This is an interesting way for an agent to adapt over time without changing its model parameters. Note that the “recurring tasks” here plays a vital reason towards the evaluation weakness.
  • Empirical results. LOTS generally improves performance over using the full tool registry across several models and benchmarks. The results also show that exposing an agent to too many irrelevant tools can noticeably hurt its performance.
  • Efficiency benefits. Once a tool space has been learned, it can be reused without repeatedly searching or ranking the full tool registry. The reduction in tool context and repeated selection cost makes the approach useful for recurring tasks.
  • Reuse across models. The paper shows that a tool space learned using one model can often be reused by another model. This suggests that some of the learned tool preferences capture properties of the task rather than being specific to one model.
Critical weaknesses
  • Unfair Experiment Comparison. LOTS learns task-specific tool preferences from multiple in-distribution fitting requests and their execution trajectories. In practice, this gives LOTS prior knowledge about which tools are likely to be useful for future requests from the same topic and domain. Most baselines mainly rely on the current request and tool descriptions. It is therefore unclear how much of the improvement comes from LOTS itself versus simply having access to historical task experience.
  • Lacking fine-tuned baselines. The retrieval and LLM-router baselines could make better use of the same fitting data available to LOTS. For example, a dense retriever could be adapted using request–tool pairs from previous trajectories, while the LLM router could use similar past trajectories as few-shot examples. The cross-model reuse experiment has a similar issue, since it transfers a tool selection that has already been fitted to the target task distribution. Comparing against similarly adapted retrieval and routing methods would better isolate the benefit of LOTS's likelihood-based scoring.
  • Evaluation remains largely in-distribution. The across-task and continual experiments use different task types, but these tasks still come from the same benchmark environments and largely share the same tool sets and task/domain structure. As a result, these experiments mainly show adaptation within a familiar distribution rather than generalization to genuinely new settings. It would be useful to evaluate on shifted test distributions, especially cases where tools that were rarely useful during fitting become important at test time.
  • Task identity (i.e., topic and domains). LOTS maintains a separate tool space for each recurring task type. In the benchmarks, these task types are already available. In a real deployment, the system would first need to decide whether a new request belongs to an existing task or requires a new tool space. Evaluating LOTS together with automatic task identification or routing would make the setting more realistic.
Questions for authors

See weakness

4 comments
AuthorsOpenReview ↗

Part 1/3 — Response to Reviewer u4QA

Thank you for recognizing the simplicity of output-based scoring, the value of persistent tool spaces, and cross-model reuse. We added controls to test your central concern: whether the gains come from likelihood scoring or simply from access to historical task experience. LOTS studies tool-space adaptation for recurring tasks with a fixed language model, where past executions are used to build a reusable configuration for future requests of the same task type. Below, we separate (i) the benefit of having history from (ii) the contribution of the likelihood-based scoring rule.

----- W1. Does LOTS benefit from historical experience that the baselines do not receive? -----

We now compare against history-based selectors using the same fitting requests and matched tool budgets. LOTS outperforms the evaluated history-frequency selectors on all three benchmarks, and pooled routing on TGB and GTA-Atomic. These comparisons show that access to historical experience alone does not account for the observed gains.

Benchmark (held-out)History-based selectorSelector (%)LOTS (%)
TGB, 8 modelsHistory-frequency selector74.8479.30
TGB, 8 modelsPooled router46.9379.30
BFCL, 5 modelsHistory-frequency selector17.6021.37
GTA-Atomic, 7 modelsHistory-frequency selector17.9718.95
GTA-Atomic, 7 modelsPooled router17.4618.95

History-frequency selector denotes counting-based selection from the same execution histories used by LOTS. The counting rule differs across benchmarks: TGB uses output-variation frequency, BFCL uses tool-call frequency, and GTA-Atomic counts calls that return non-empty outputs. Non-empty outputs are not treated as evidence of correctness. These variants share the principle of ranking tools by historical occurrence statistics, while LOTS scores the contribution of their observed outputs to the fixed response. LOTS exceeds output-variation frequency on all eight TGB models and usage frequency on all five BFCL models; on GTA the margins are smaller.

(b) Same traces, different scoring rule. The trace-judge control (Table 17) holds fixed everything except the scoring rule: the executed traces, frozen responses, task-level aggregation, and budget. It replaces likelihood attribution with an LLM's explicit judgment of which tool outputs support the response. LOTS outperforms it on all five models, which isolates the contribution of the scoring mechanism from access to historical experience.

(c) Existing task-level baselines. Dense and Tool2Vec already build task-level rankings from the fitting requests and reuse them at evaluation. TTO and Beam Search also execute candidate spaces on the fitting requests and select by fitting accuracy, i.e., with correctness labels. Only the per-request LLM router and Jev rely solely on the current request; we will separate the two groups clearly in the main text.

We will incorporate the matched-budget history-based selectors into the main results, state the information each baseline receives, and present the trace-judge comparison as the primary evidence for the scoring rule.

AuthorsOpenReview ↗

Part 2/3 — Response to Reviewer u4QA

----- W2. Fine-tuned retrieval, history-conditioned routing, and cross-model reuse -----

Request-level retrieval and routing select tools for each incoming request. ToolLLM (Qin et al., 2023), for example, trains a retriever on instruction–API relevance pairs, while Tool2Vec (Moon et al., 2024) constructs usage-driven tool representations. Task-level configuration is another established setting: OctoTools (Lu et al., 2025) selects task-specific toolsets using validation execution performance without updating model parameters. Our comparisons cover both settings, including retrieval and LLM routing, as well as TTO and Beam Search, which already use task-specific execution feedback and fitting-set correctness labels. Historical experience is therefore not exclusive to LOTS.

LOTS constructs and updates a persistent task-level tool space from execution experience while keeping model parameters fixed. It scores how removing a tool’s observed output changes the likelihood of a fixed response, conditional on the remaining evidence, and aggregates this contribution across requests. The history-based selectors control access to experience, while the trace-judge comparison in Appendix E.2 controls the executed traces, frozen responses, aggregation, and tool budget, changing only the attribution rule. These comparisons directly address whether the gain comes merely from historical information. Cross-model reuse tests a complementary property: whether a configuration constructed for a task remains useful when the serving model changes. Our held-out evaluation tests this reuse on requests excluded from configuration fitting.

Fine-tuning tradeoffs. A fine-tuned retriever can condition its selections on different requests, but this does not by itself provide ongoing adaptation to new execution evidence. Once its parameters are fixed, incorporating that evidence requires an additional update mechanism, such as retraining or an external memory. Fine-tuning also introduces training-data preparation and parameter-optimization costs. Repeated retraining adds further cost when adaptation is needed. Moon et al. (2024) similarly identify retraining, domain-specific training data, and LLM-selection latency as practical concerns. LOTS instead updates explicit tool scores and reuses the resulting configuration without a separate retrieval or routing call for every request. Therefore, a fine-tuned retriever test an additional adaptation strategy.

References

[1] Lu et al. OctoTools: An Agentic Framework with Extensible Tools for Complex Reasoning. ACL 2026 Oral.

[2] Qin et al. (2023). ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs. ICLR 2024.

[3] Moon et al. (2024). Efficient and Scalable Estimation of Tool Representations in Vector Space. arxiv.

AuthorsOpenReview ↗

Part 3/3 — Response to Reviewer u4QA

----- W3. Shifted test distributions, including tools rarely useful during fitting -----

We added an experiment in which tools pruned during fitting become required at test time.

On five TGB models, the task identity is held fixed while the required tool set changes from round 4, so some newly required tools were pruned from the fitted space. With exploration, all 27 cells missing a required tool restore it, 25 of them within 40 requests, and mean post-change success is 68.2% at ε=0.1 and 70.6% at ε=0.25. Without exploration, recovery is rare. A pruned tool can re-enter only if it is exposed again, so exploration is necessary. Our recovery claims are limited to the evaluated abrupt shifts.

We agree that all evaluations remain within the benchmark environments. By "self-evolution" we mean updating an external tool configuration from accumulated experience with fixed model parameters. We do not claim open-ended continual learning or robustness to arbitrary cross-domain shift, and we will remove wording that suggests otherwise.

We will incorporate the changing-requirement experiment into the paper, define tool-space evolution explicitly, and distinguish held-out request generalization, adaptation to changing tool demand, and cross-domain transfer.


----- W4. Task identity in real deployment -----

Task groups can be obtained from deployment metadata or inferred through text clustering; LOTS does not require manually annotated task identities.

When metadata is available, requests can be grouped by workflow, application, or API domain. These identifiers provide the recurring-task context used to construct and reuse a tool configuration.

When such metadata is unavailable, requests can instead be grouped through text-only clustering. In our four-model TGB evaluation, inferred grouping matches the performance obtained with oracle task identities. Separate clustering diagnostics also recover the TGB task families and BFCL API classes, supporting the feasibility of inferring groups in these settings.

We will clarify these two ways of supplying task groups and include the inferred-grouping results. Grouping is an input to LOTS; text-based grouping can still be ambiguous when semantically similar requests require different tools.

Dear Reviewer u4QA,

As the author–reviewer discussion period draws to a close on Nov. 18, we would appreciate any further thoughts you may have on whether our response addresses your main concerns.

Thank you again for your time and consideration!

Reviewer_bQBCOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

LOTS builds persistent task-specific tool configurations. Each tool is scored by how much the likelihood of a frozen model answer drops when that tool's evidence is removed. Scores are averaged per task, and the top-K tools (on BFCL, the top-K documented tools) are kept. Experiments cover TGB, BFCL and GTA-Atomic, plus sequential updates and cross-model reuse. Scoring needs no labels, but K is selected using fitting labels.

Critical strengths
  1. Simple, practical adaptation. The model stays frozen and configurations are reused. LOTS beats Keep all on all eight TGB models while keeping 2.8–6.8 of 15 tools, and on all five BFCL models (Tables 1–2).

  2. Useful controls. Fixed-budget comparisons separate ranking quality from budget selection (Table 9). LOTS beats an LLM trace judge on all five models (Table 17), and evidence-only deletion nearly matches re-execution (Table 18).

Critical weaknesses

Post-rebuttal update: I am raising my rating to weak accept. The matched history-based baselines and held-out transfer results substantially address my main concern about likelihood scoring. GTA gains are modest, but the context savings remain useful. The full cost of continued updating and BFCL shared-message deletion remain limitations. I find the narrower tool-selection and reuse contribution sufficient for weak acceptance.

Original assessment:

  1. The advantage over cheaper usage signals is unclear. TGB runs every served tool automatically, so it primarily tests filtering precomputed tool evidence, rather than the model's ability to choose and execute tools. BFCL prunes only documentation, yet no task-level call-frequency baseline is reported. On GTA read_arith with Qwen2.5-7B, pooling the router's per-request choices already gives LOTS's Calculator+OCR pair (Appendix E.2). Suggestion: compare against these selectors on held-out requests at matched K.

  2. The value of continued updating is not established. Under the stationary-task setting, updating reaches 0.757 versus 0.750 when the space is frozen after the first batch (Appendix D.5). The cold-start gains are compared only with random and union updaters (Table 13). The payback figures leave out trace collection and later updates (Table 20, Figure 20), and BFCL payback already requires 526–1,244 records under the reported accounting. Suggestion: report cumulative accuracy and total cost (Eq. 18) against a stronger updater under the same exploration schedule.

Minor Weaknesses

  1. Figure 6 fits and evaluates on the same 101 records, so Table 6 should be the primary transfer evidence.
  2. BFCL deletion removes whole multi-tool messages, affecting up to 44.1% of (record, tool) pairs (Appendix B.2).
  3. GTA's Keep-all menu includes two unavailable tools, so some of the reported token savings may come from removing unavailable tools. A Keep-all-available baseline is needed.

The results are promising, but the paper has not yet shown that likelihood attribution beats cheaper usage signals, or that continued updating adds value. Matched comparisons could move me toward weak acceptance.

Questions for authors
  1. Does LOTS beat call frequency on BFCL and pooled-router selection on GTA at matched K?
  2. Does continued updating beat a frozen space once all costs are counted?
  3. How do BFCL results change if single calls are deleted instead of whole messages?
5 comments
AuthorsOpenReview ↗

Part 1/3 — Response to Reviewer bQBC

Thank you for the detailed review and for recognizing the practical value of persistent tool configurations and the existing controls. We address the contribution of likelihood scoring separately from the benefit of continued updating, using additional history-based comparisons and analyses.

----- W1 / Q1. Does likelihood attribution outperform cheaper usage signals at the same tool budget? -----

Thank you to the reviewer for raising this point. We added selectors that use the same fitting requests and the same execution histories. The comparisons below match the retained tool budget within each benchmark.

On TGB, we evaluated on eight models with task-specific matched budgets and 2,880 held-out requests per model. We compare with an output-variation frequency baseline, which ranks tools by the share of fitting requests on which their outputs change. This is a TGB-specific historical signal, distinct from ordinary call frequency. A pooled router aggregates per-request selections into a persistent task-level menu.

ModelCalled at least onceOutput-variation frequencyPooled routerLOTS
Mistral-7B13.4160.7537.5065.78
Qwen3-4B23.0056.9733.0968.66
Qwen3.5-9B25.5361.2235.1265.53
Qwen3-8B9.9777.5047.0381.66
Llama-3.1-8B40.2278.8155.5986.06
Qwen2.5-7B31.2280.9147.5982.84
Phi-439.2590.9451.4492.16
Qwen2.5-14B37.1691.6268.1091.72
Mean over eight models27.4774.8446.9379.30

LOTS exceeds output-variation frequency on all eight models, with a mean improvement of 4.46 percentage points.

On GTA-Atomic, we evaluated seven models on 93 held-out requests with three executions.

SelectorMean success (%)
Keep all16.23
Raw call count16.39
Non-empty-output call count17.97
Pooled router17.46
Per-request router14.59
LOTS18.95

LOTS has the highest observed mean. The read_arith example is a useful boundary case. There is nevertheless a distinction in the information used to rank tools. In the read_arith fitting history, the router selects Calculator ten times and both OCR and Solver three times, leaving a frequency tie for the second slot. LOTS assigns OCR a score of 0.171 and Solver a score of −0.012. This illustrates how output attribution can distinguish candidates that a frequency statistic alone cannot.

On BFCL, we evaluated five models with 160 evaluation records per model. Under the submitted protocol, LOTS averages 21.37%, compared with 17.60% for usage frequency, and is higher on all five models.

We will incorporate these historical baselines into the paper, distinguish the benchmark protocols, and state where the observed advantage is large, modest, or inconsistent across implementations.

AuthorsOpenReview ↗

Part 2/3 — Response to Reviewer bQBC

----- W2 / Q2. Does continued updating improve on a frozen space once all costs are counted? -----

The value of continued updating depends on whether the initial tool space has already captured the task's requirements. Our paper evaluates both an initially incomplete space and a space fitted with broad initial tool exposure.

Figure 4 examines the former setting: the registry contains 120 tools, but each request initially exposes at most 16. Even with a fixed task distribution, the initial configuration can therefore miss necessary tools. Subsequent exploration provides new output evidence, and the updates incorporate useful tools into the persistent space. The accuracy increases occur when these updates complete a required tool chain. The figure reports improvement on fixed evaluation requests, including the mean trajectory over all 24 runs per model, rather than only selected successful examples.

The frozen-space comparison in Appendix D.5 addresses a different condition. With the smaller 15-tool registry, the first batch already supplies much of the information needed to construct the space. Limited improvement after that initial fit is therefore compatible with the sustained gains observed under incomplete initial exposure. A stationary task distribution does not imply that the agent has already identified its useful tools. Table 13 also compares update rules under the same request order, exploration probability, and seeds, examining how newly observed evidence is consolidated into the serving space.

We will make this distinction more explicit: continued updating supports discovery and refinement when the initial space is incomplete, while persistence enables reuse once a useful configuration has been established. Its accuracy benefit and its cost-effectiveness are separate questions; the reported payback estimates remain conditional on their stated accounting assumptions.

We will incorporate the stationary comparison and its accounting scope into the paper, attribute the stable-stream benefit to configuration reuse, and discuss continued adaptation using the changing-demand experiment separately.


----- Minor W1. Figure 6 fits and evaluates on the same 101 records. -----

Figure 6 used a separate 101-record BFCL tune set for cross-model configuration and budget diagnostics, keeping the requests fixed across model pairs. To evaluate reuse on unseen requests, we conducted an additional TGB experiment following Figure 6’s source-model × deployment-model design, with disjoint fitting and evaluation records.

We use eight models and ten TGB task families. Each family contains 400 requests, split into 80 fitting requests and 320 held-out evaluation requests, with the same split across models. For each source–deployment pair (s,d)(s,d), LOTS constructs a task-specific tool ranking from source model ss’s execution traces on the 80 fitting requests. Deployment model dd then serves the 320 held-out requests using the selected tools from that ranking, without recomputing the ranking from its own traces.

As in Figure 6, rows correspond to deployment models and columns to source models. We evaluate all 8×88\times8 ordered pairs: the eight diagonal entries use the deployment model’s own ranking, while the 56 off-diagonal entries use another model’s ranking. We compare two budget rules. Under deployment-fitted K⋆K^\star, each deployment model’s task-specific budget is selected on its fitting split and held fixed across all source rankings in that row. Under fixed K=4K=4, every source–deployment pair retains four tools per task. Own and transferred rankings are compared at the same budget.

Budget ruleOwn configuration (%)Transferred configuration (%)Own minus transferred (pp)
Deployment model’s fitted K⋆K^\star79.3075.87+3.43
Fixed K=4K=468.2968.33−0.04

The table summarizes diagonal and off-diagonal performance. Model-specific rankings perform better on average under deployment-fitted budgets, while the aggregate difference is negligible at fixed K=4K=4.

We will replace the same-record analysis in Figure 6 with this held-out transfer matrix. The experiment evaluates cross-model reuse on unseen requests within the same task families.

AuthorsOpenReview ↗

Part 3/3 — Response to Reviewer bQBC

----- Minor W2 / Q3. What changes when individual tool results are deleted instead of whole multi-tool messages? -----

Appendix B.2 (p. 18) already specifies the benchmark-specific deletion operations, and Appendix E.3 (pp. 34–35, Table 18) compares chain re-execution with individual-output deletion on TGB. On BFCL, the additional controls below test sensitivity to repeated occurrences and deletion size.

As described in Appendix B.2, TGB records one line per served tool. The default intervention removes the candidate tool and re-executes the chain, so dependent downstream outputs can change while unrelated outputs remain unchanged. Appendix E.3 explicitly compares this with an evidence-only intervention that deletes just the candidate's recorded output line while retaining the other recorded outputs. In Table 18, both operations recover the complete chain in all nine tool-using families; evaluation accuracy is identical at K=4K=4 and K=8K=8 and differs by 0.2 percentage points at K=2K=2 (Qwen2.5-7B).

Appendix B.2 also states that GTA-Atomic deletes only the candidate's output line, whereas BFCL removes the whole assistant message and all observations of the step when that message calls multiple tools. It reports that 24.9% of BFCL tool-calling messages involve multiple distinct tools and that 44.1% of (record, called tool) pairs involve at least one such message. Thus, shared-message deletion is an explicitly documented limitation of the BFCL implementation. To further examine sensitivity to repeated occurrences and deletion size, we evaluated the following occurrence-level variants on five BFCL models at K=16K=16, with 40 evaluation records per class under the demotion protocol.

Deletion / aggregation ruleMean held-out success (%)
All occurrences of the tool, submitted score19.88
Score divided by occurrence count19.38
Score divided by characters removed19.88
Largest single-occurrence marginal20.25
Within-record rank normalization16.63

The occurrence-count, character-count, and single-occurrence variants remain close to the submitted score in average accuracy, despite changes in the selected menus. Rank normalization performs worse in this comparison. These controls address sensitivity to repetition and deletion size; they do not remove the possible confounding from deleting other tools' results in the same BFCL message.

We will incorporate these additional BFCL controls into the paper and make the existing descriptions in Appendix B.2 and the TGB comparison in Appendix E.3 more prominent, while retaining shared-message deletion as a limitation of the BFCL attribution analysis.


----- Minor W3. Do GTA token savings partly come from removing unavailable tools? -----

Substantial compression remains when the reference menu contains only available tools. The accuracy difference becomes smaller. We added Keep-all-available, which removes the unavailable tools from the corresponding task menus.

GTA menuMean success (%)Tool-schema tokens per request
Original Keep all16.232,138–2,188
Keep-all-available18.131,955–2,003
LOTS18.95224–515

The evaluation covers seven models, 93 held-out requests, and three executions. Relative to Keep-all-available, LOTS reduces schema tokens by 74–89% and total prompt tokens across ReAct turns by 32–73%.

We will incorporate Keep-all-available as the primary GTA reference and separate reductions in unavailable tools from compression of the executable menu.

Comments on the paper

1 comment

We thank all reviewers for their feedback. We briefly summarize the main evidence added during the discussion:

Contribution of likelihood scoring. With the same fitting histories and matched tool budgets, LOTS outperforms history-frequency selectors on TGB (79.30% vs. 74.84%), BFCL (21.37% vs. 17.60%), and GTA-Atomic (18.95% vs. 17.97%).

Tool selection without correctness labels. Scoring requires no correctness labels; the submitted task-specific budget selection does. With a fixed budget of K = 3, LOTS achieves 71.50% on TGB versus 65.50% for Keep all, retaining a benefit without label-based budget selection.

Adaptation and cost. Under the evaluated changes in tool requirements, exploration restores previously pruned required tools in all 27 cases. In the additional cost comparison, updating improves accuracy on the new demand from 22.13% to 69.59% and stream efficiency from 262 to 440 correct answers per million tokens, accounting for serving, exploration, scoring, and update.

Reuse and context reduction. Held-out cross-model experiments support configuration reuse. Against Keep-all-available on GTA, LOTS achieves 18.95% versus 18.13% success while reducing tool-schema tokens by 74–89%.

These results support likelihood-guided tool selection and persistent configuration reuse, with adaptation demonstrated under the tested shifts. We will incorporate the additional evidence and clarify the scope of our claims.

Thank you again for your time and consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.