acceptodds
Under review as a conference paper at ICLR 2027

When Should Agents Build Reusable Tools?

Abstract

Modern agents can improve over time by turning solved tasks into persistent reusable tools, but building such capabilities requires an upfront investment that may not pay off. We study when an online agent should build a new tool rather than patch the current task or reuse an existing one, under uncertain task recurrence and stochastic tool quality. We formulate this as cost-sensitive capability acquisition and develop Demand-Sensitive Acquisition (DSA), a controller that combines recurrence estimates with uncertainty in current success, durable admission, and future reuse. In a separable model where each tool serves one recurring task type, we derive the exact finite-horizon optimal policy and decompose the cost gap of building when a task type first appears; on an executable API-migration benchmark with 108 LLM-generated candidates, DSA lowers modeled cost by 6.7% over 200 heavy-tailed task sequences, relative to a pattern-level baseline that builds when each pattern first appears. These results frame agent self-improvement as a sequential investment problem over reusable capabilities.

Primary area:
foundation or frontier models, including LLMs
Keywords:
self-evolving agents, reusable tool acquisition, sequential decision making

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_QriqOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper studies when an agent should build a reusable tool, patch the current task, or reuse an existing tool. It derives an exact finite-horizon acquisition rule in a known-parameter separable model and proposes Demand-Sensitive Acquisition (DSA), which estimates task recurrence, acquisition quality, and reuse reliability. On an API-migration benchmark with 108 frozen generated candidates, DSA reduces modeled cost relative to Pattern-First under heavy-tailed demand, although the comparison reverses under balanced demand.

Critical strengths

(1) The problem is practical, and the analysis gives a useful account of acquisition timing and retry. Separating success on the current task from admission for future reuse is also well motivated.

(2) The controlled evaluation is carefully documented. Policies share candidate queues and service rules, and the paper reports paired uncertainty, ablations, cost sensitivity, and unfavorable results. The executable artifact makes these comparisons easier to check.

Critical weaknesses

(1) The theoretical discussion takes up too much space relative to the empirical validation. I would shorten the main-text analysis and use the space to present stronger experimental evidence.

(2) Experimental coverage is limited to one API-migration benchmark with relatively simple local transformations, such as function and keyword renaming. This makes it difficult to assess the method's value for more demanding agent tasks.

(3) The 108 candidate tools are generated in advance and frozen. During evaluation, the agent tries and validates pre-generated candidates, without generating or refining tools in response to execution feedback. This tests acquisition decisions over a fixed pool, leaving dynamic tool generation untested.

Questions for authors

Please refer to the weakness section.

Reviewer_rFA9OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates when AI agents should build reusable tools rather than repeatedly solving tasks or reusing existing capabilities. It proposes Demand-Sensitive Acquisition (DSA), an online controller that estimates future task demand, tool quality, and reuse benefits to make cost-aware acquisition decisions. The authors further derive an exact optimal acquisition policy under a simplified model. Experiments on an executable API-migration benchmark show that DSA reduces cumulative modeled cost by 6.69% compared with Pattern-First under heavy-tailed task recurrence.

Critical strengths
  1. Meaningful problem formulation. This work reframes LLM agent tool construction as an investment decision problem. Beyond tool generation and reuse, it explicitly considers the trade-off between upfront acquisition costs and long-term reuse benefits.
  2. Interpretable theoretical contributions.The paper derives an exact finite-horizon optimal acquisition policy under a separable stochastic-admission model and identifies two sources of inefficiency in the Build-First strategy. These results provide theoretical justification for acquisition timing decisions.
Critical weaknesses
  1. Limited experimental validation and generalizability. The evaluation primarily compares DSA against relatively simple heuristic baselines, such as Family-First, Pattern-First, and Always-Build-When-Eligible, without direct comparisons to existing adaptive or cost-aware capability acquisition methods. This makes it difficult to establish DSA's advantages over more competitive approaches. Furthermore, the experiments rely on a highly controlled API-migration benchmark with a pre-generated, frozen candidate pool and predefined modeled costs, which do not fully capture online tool generation, evolving task demand, or actual execution overhead in realistic agent systems. The evaluation also provides limited analysis of how online posterior updates influence acquisition decisions. These limitations weaken the empirical evidence for DSA's practical effectiveness and generalizability.
  2. Gap between idealized theoretical assumptions and practical settings. The theoretical analysis relies on strong simplifying assumptions, but realistic agent environments involve uncertain demand, heterogeneous tool quality, and potentially overlapping or interdependent capabilities. Although Section 5.4 acknowledges the differences between the theoretical model and the executable setting, the paper does not sufficiently establish whether the derived optimal acquisition rule remains effective when these assumptions are relaxed. This limits the extent to which the theoretical guarantees can justify acquisition decisions in realistic agent settings.
Questions for authors
  1. The current experiments mainly compare against simple fixed-rule baselines. Have the authors compared DSA with existing cost-aware or adaptive tool-acquisition methods? If not, how can the advantage of DSA over these stronger approaches be established?
  2. The evaluation uses a pre-generated, frozen candidate pool and normalized modeled costs. To what extent do these controlled settings affect the reported advantages of DSA? Would the relative performance remain consistent under online tool generation or empirically measured generation and validation costs?
  3. The optimality result relies on separable task types and known parameters, whereas the practical DSA controller operates under demand uncertainty and cross-pattern tool transfer. Although Section 5.4 acknowledges these differences, how closely does DSA approximate optimal acquisition decisions when these assumptions are relaxed? Could the authors provide theoretical insights or empirical comparisons against an oracle policy to clarify this gap?
Reviewer_fCWwOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This paper studies the problem of agent decision-making around when to build a persistent tool. The authors formulate this problem as cost-sensitive capability acquisition that distinguishes between tool quality and projected future reusability, accounting for the cost incurred and saved from tool building and reuse. The authors introduce Demand-Sensitive Acquisition (DSA), a controller that estimates when persistent tool building is worthwhile by estimating cost saved from expected reuse of a persistent tool and upfront cost of tool building itself. The approach is evaluated on a synthetic API migration benchmark introduced by the authors.

Critical strengths

This paper studies an important and underexplored problem of persistent tool building, formulating it as an investment decision that needs to account not only for candidate tool quality but the trade-off between the upfront cost of tool building and expected future reusability. Separation between current task success (G) and validation for durable reuse (I) is well motivated and provides a useful distinction between immediate utility and estimated future value.

Critical weaknesses
  • The main method (DSA) is only evaluated on proposed synthetic benchmark, which assumes a pre-defined problem structure that is unlikely to be known in realistic settings. The benchmarks provides task family and pattern labels as ground truth and assumes that task taxonomy is fixed. In practice, inferring which tasks belong to the same reusable type is itself a challenging part of the problem. The paper also contains no details on example task family, client, tool candidate, or LLM generation prompt.
  • Estimating the cost incurred and saved from creation of a persistent tool is itself a challenging problem, but they are treated as fixed constants. Table 7 in the appendix also shows that each methods have varying sensitivity to build cost. The results are also mixed based on cost. DSA incurs least cost under heavy-tailed demand, but is less cost-effective than Pattern-First under balanced demand. While some justification is provided in the appendix, the authors don’t test this explanation. Given these results and cost sensitivity provided in Appendix B.7 I doubt the practical value of DSA in deployment settings where cost estimation itself is often challenging.
Questions for authors
  • Could you provide more detail on benchmark construction, including examples of task families, clients, tool candidates, and LLM generation prompts?
  • Could you comment on how DSA might be deployed in a realistic setting, where task families should be inferred and tool creation costs are not known? Have you considered evaluating beyond the synthetic benchmark, for example in a real tool-use setting?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.