- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper studies when an agent should build a reusable tool, patch the current task, or reuse an existing tool. It derives an exact finite-horizon acquisition rule in a known-parameter separable model and proposes Demand-Sensitive Acquisition (DSA), which estimates task recurrence, acquisition quality, and reuse reliability. On an API-migration benchmark with 108 frozen generated candidates, DSA reduces modeled cost relative to Pattern-First under heavy-tailed demand, although the comparison reverses under balanced demand.
Critical strengths
(1) The problem is practical, and the analysis gives a useful account of acquisition timing and retry. Separating success on the current task from admission for future reuse is also well motivated.
(2) The controlled evaluation is carefully documented. Policies share candidate queues and service rules, and the paper reports paired uncertainty, ablations, cost sensitivity, and unfavorable results. The executable artifact makes these comparisons easier to check.
Critical weaknesses
(1) The theoretical discussion takes up too much space relative to the empirical validation. I would shorten the main-text analysis and use the space to present stronger experimental evidence.
(2) Experimental coverage is limited to one API-migration benchmark with relatively simple local transformations, such as function and keyword renaming. This makes it difficult to assess the method's value for more demanding agent tasks.
(3) The 108 candidate tools are generated in advance and frozen. During evaluation, the agent tries and validates pre-generated candidates, without generating or refining tools in response to execution feedback. This tests acquisition decisions over a fixed pool, leaving dynamic tool generation untested.
Questions for authors
Please refer to the weakness section.