acceptodds
Withdrawn from ICLR 2027

TASL: Adapter Library for Evolving Tools

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Tool-augmented language agents extend LLM capabilities by calling external tools, but API evolution can invalidate previously successful calls. Tool calls therefore need to be adapted to match the updated interfaces. Existing methods that rely on model generation to adapt calls still require repeated model inference for new calls affected by the same API change. To reduce repeated adaptation, we introduce TASL (Tool-Adapter Synthesis Library), a training-free framework that synthesizes persistent executable adapters for evolving APIs. TASL turns each API change into a reusable adapter, validating it via execution and round-trip checks so that subsequent calls apply the stored program directly, with no adaptation-related model inference. Model parameters remain fixed, and no task-answer supervision is used. Experiments across 7 domains, 11 tools, and 2 evolved API sets in ToolQA-D show that TASL achieves higher accuracy than all evaluated training-free baselines while using fewer adaptation-related LLM calls. Specifically, it improves mean exact-match accuracy by 13.2 and 7.2 percentage points over the strongest baseline with 4B and 14B backbones, respectively. Code will be released after acceptance.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Tool-Augmented Language Agents, API Evolution, Adapter Synthesis, Training-Free Adaptation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_NYhJOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

TASL proposes a training-free API-evolution adaptation layer that synthesizes executable argument-transformation adapters from deprecation feedback, validates them via environment acceptance and round-trip consistency, and persistently reuses them across affected calls; on ToolQA-D with Qwen3-4B/14B, it outperforms training-free baselines and reduces adaptation-related LLM calls, while remaining competitive with a training-based ToolEVO baseline.

Critical strengths
  • The core idea is simple and practical: synthesize once, validate, cache, and reuse executable adapters so successful cache hits require no adaptation-related LLM inference.

  • The validation protocol combines environment acceptance with round-trip checks on current and historical inputs, and the lossy-adapter case study clearly demonstrates its value.

  • The experiments are reasonably systematic, covering two backbones, two evolved API versions, multiple baselines, ablations, question-order sensitivity, hyperparameter sensitivity, and cost statistics.

Critical weaknesses
  • External validity is limited to ToolQA-D’s fixed and largely programmatic API perturbations, with no evidence on return-schema changes, authentication, pagination, semantic changes, or multi-step dependencies.

  • Failure handling and statistics are under-analyzed: synthesis failure terminates the episode without fallback, and the main results lack confidence intervals, variance estimates, and adaptation success/eviction/cache-hit rates.

  • Baseline fairness and strength need clarification, since Prompt-Adapt and Schema-Adapt are author-constructed and ReAct/Reflexion do not share adaptation memory across questions.

  • The paper has noticeable presentation issues, including duplicated contribution lines, repeated table captions, malformed algorithm text, garbled tool names, and code-snippet errors in the appendix.

Questions for authors

None

Reviewer_ziisOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes TASL, a training-free framework for adapting tool calls to evolving APIs in tool-augmented question answering. TASL synthesizes executable argument transformations from interface feedback, validates them through execution and round-trip checks, and stores accepted adapters for reuse across tasks. This design aims to detect information loss while avoiding repeated model-based repairs. Experiments on ToolQA-D with 4B and 14B backbones show higher accuracy and fewer adaptation-related LLM calls than the evaluated training-free baselines, with ablations supporting the contributions of validation and persistent reuse.

Critical strengths

1: The paper explores reusing executable adapters to improve the reliability and efficiency of tool use under API changes. This addresses a practical limitation of repeated model-based repair: different tasks often encounter the same interface change, yet the model must derive the correction again. Retaining the transformation as a program offers a straightforward way to avoid this redundant work.

2: The validation mechanism goes beyond checking whether an adapted call executes successfully. Round-trip checks help detect dropped arguments that execution alone may overlook, while historical inputs make validation less dependent on the first observed call. Together, these provide a useful admission criterion without requiring task-answer labels.

3: The empirical results show consistent improvements across both model sizes, with mean accuracy gains of 13.2 and 7.2 percentage points over the strongest training-free baseline. TASL also reduces total LLM calls and token usage. The ablations support the usefulness of the main components, and the question-order experiments provide additional evidence that the results are reasonably stable.

Critical weaknesses

W1: The technical novelty relative to existing executable reuse methods remains insufficiently established. Section 2 discusses related systems such as CRAFT, Voyager, and TTHE, but primarily distinguishes TASL by its application to API argument transformations. The paper should explain more clearly whether its synthesis, validation, and memory mechanisms introduce substantive differences from prior approaches, rather than mainly combining established ideas for a new setting.

W2: The validation checks do not establish that an adapted call preserves the intended semantics. In Section 3.3, a transformation could map arguments to incorrect fields while still passing execution and round-trip checks, provided its inverse reverses the mapping. The ablation in Section 4.4 demonstrates that validation improves accuracy, but does not show how often incorrect adapters still pass these checks. An analysis of such failures would help assess the reliability of persistent reuse.

W3: The experimental setting leaves the generality of the method unclear. Section 4.1 evaluates only ToolQA-D, where deprecation warnings supply the target tool name, parameter list, and an argument example. It remains unclear how well TASL works when feedback is incomplete or API changes require more than argument transformations. The comparisons also lack a closely related executable reuse baseline, making it difficult to assess TASL’s advantage over existing approaches with similar reuse capabilities.

Questions for authors
  1. What technical differences distinguish TASL from prior executable-reuse methods such as CRAFT, Voyager, and TTHE, beyond its application to API adaptation?
  2. How often do adapters pass execution and round-trip validation but still produce semantically incorrect calls? Were such failures observed in your experiments?
  3. How dependent are TASL’s gains on warnings supplying the target API signature and argument example? Do you have evidence of robustness to incomplete feedback?
Reviewer_4JB6OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes TASL, a training-free layer between a frozen agent and its tools for handling API changes. When an outdated call fails and the environment returns a deprecation warning, TASL asks the LLM to write a Python adapt function, which maps the old argument dictionary to the new one, and an inverse_adapt function. A candidate is accepted only if two checks pass: the new API accepts the transformed call, and the inverse reconstructs the current input and a few earlier ones (round-trip consistency). Accepted adapters are cached under the key (old tool, new signature) and applied to later calls with no LLM involvement. Adapters whose success rate falls below a threshold are evicted. On ToolQA-D (7 domains, 11 tools, 2 evolved API versions) with Qwen3-4B and 14B, TASL beats ReAct, Reflexion and two caching baselines built by the authors (Prompt-Adapt and Schema-Adapt) by about 13 and 7 points, and makes far fewer repair calls. On a Qwen3-4B fine-tuned for tool use, it also edges out ToolEVO.

Critical strengths

Storing a verified executable transformation, instead of re-deriving the fix on every call or caching text for the model to re-read, is a simple idea that suits a stable API change. The round-trip check is a cheap validation criterion that needs no labels and catches a real failure mode: the FilterDB passes the environment but silently drops a condition. The ablation shows that caching drives most of the gain.

Critical weaknesses

[W1] The setting hands the method most of the answer, and it is too narrow to show that the approach generalizes. In ToolQA-D, the deprecation warning gives the new tool name and an example of the new arguments (seen in Appendix E). TASL reads it with regexes written for this benchmark's warning format (also seen in Appendix H), and if the regex does not match, no adaptation is attempted. What remains is an argument-level rewrite, and Table 9 shows that every change is a rename, split, merge or fixed prefix produced by one synthetic generator. Real API evolution is often less explicit: errors with no migration hint, changed semantics or units, changed response formats, or removed functionality. None of these is tested, and an argument-only adapter cannot handle response-side changes at all. Round-trip consistency checks only that no input information is lost, not that the call is correct. For newly required parameters with no old counterpart (return_num, passage_num, ReturnValue), the adapter hard-codes a guessed default that neither check can validate. With one benchmark, 11 tools and 2 versions, the evidence does not support the general claim about evolving tools.

[W2] The baselines do not isolate what LLM program synthesis adds. Prompt-Adapt and Schema-Adapt are constructed for this paper, and ReAct and Reflexion keep no memory across questions, so the large repair-call savings come partly from comparing against methods that cannot reuse anything. Against the caching baselines, total LLM calls fall by only about 15–25%. Schema-Adapt, which gives the model the updated schema, scores about 10 points below plain ReAct on V1 at both scales. This is surprising, since the new schema should help, and it suggests the baseline is not well tuned. The natural cheap baselines are missing:

  • ReAct with memory across questions: keep the raw warning, or the first corrected call that succeeded, in the prompt for later questions in the same group. This directly tests whether memory alone, rather than synthesized code, explains the gain.
  • The signature parsed from the warning inserted directly into the tool description, with no LLM rewrite.
  • A rule-based key-alignment adapter derived from the warning's example.

Given how regular the Table 9 changes are, these could close much of the gap. The oracle study points the same way: plain ReAct is already about 88% API-correct, against about 98% for TASL. Only Qwen3-4B and 14B with thinking disabled are tested, so it is unclear whether a stronger agent (like GPT or Opus) would simply repair correctly from the warning. The ToolEVO comparison is a single run with a gap of under 3 points.

Questions for authors
  1. How does TASL perform when the warning gives only the new tool name, or only a generic error, with no argument example? How much of the gain depends on the example in the warning?
  2. What accuracy do the missing baselines reach: ReAct with the warning or the corrected call kept across questions, the parsed new signature inserted directly into the prompt, and a rule-based adapter built from the example? Why does Schema-Adapt fall below ReAct on V1?
  3. ReAct drops from about 41% on V1 to about 13% on V2 (4B), yet in the oracle study (Fig. 4) it makes more correct API calls on V2 than on V1. What causes the V2 collapse, if not call adaptation? Example ReAct trajectories on V2 would help.
  4. Do TASL's synthesis attempts on a cache miss use up the agent's step budget (15, or 6–8 for some domains), as ReAct's failed calls do? If not, how much of the gap remains under an equal budget?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.