acceptodds
Under review as a conference paper at ICLR 2027

Can Tool-Using Agents Keep Adapting as Tool Environments Change?

Abstract

Large language model (LLM) agents increasingly rely on external tools, yet existing tool-learning methods typically assume relatively stable tool environments. In practice, tools and their interfaces continuously evolve. We study dynamic tool learning, where an agent must retain unaffected tool-use capabilities while efficiently adapting behaviors invalidated by environmental changes. Our key observation is that multi-step tool-use behaviors naturally exhibit ordered and compositional structures. Inspired by natural selection, we propose CATE (Continual Adaptation to Tool Environments), which organizes tool-use experience as an evolving population of skills. CATE represents accumulated tool-use experience as reusable Skills that capture not only individual tool usage but also the ordering and dependencies among tools. Starting from single-tool Skills, it progressively develops compositional multi-step behaviors through interaction, recombination, and failure-driven refinement. When the environment changes, CATE localizes affected skills and adapts them through experience inheritance, preserving still-valid knowledge instead of relearning the entire tool space. To systematically evaluate continual adaptation, we introduce DynaToolBench, a controlled benchmark built on a stable MirrorAPI execution backend. DynaToolBench models tool replacement over three successive timesteps while ensuring that evaluation tasks remain solvable as the environment evolves. Experiments across evolving tool environments show that CATE maintains strong performance over time, effectively recovering capabilities affected by tool changes while largely preserving previously acquired behaviors.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
tool learning, LLM agents, dynamic tool environments, continual adaptation, tool-use skills, continual learning, agent learning

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_7nccOpenReview ↗
Rating
4(+1)
Confidence
4(+1)
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

The paper studies adaptation of tool-using agents to solution-preserving API replacements. It introduces DynaToolBench, comprising 64 short-chain tasks over 50 APIs and 30 longer-chain tasks over 34 APIs, evaluated at three timesteps. Versioned adapters change agent-visible tool names, arguments, and response fields while preserving an executable MirrorAPI backend. The proposed CATE method builds a SkillBank of atomic and pairwise composite skills through documentation-grounded expansion, interaction, and reflection. After an environment transition, it identifies affected skills, retrieves candidate replacement APIs, revises interface-dependent guidance, and transfers supported experience from retired atomic skills while retaining unaffected knowledge. Experiments use Qwen3-4B and Qwen3-8B, three repetitions, and an independent requirement-level LLM evaluator. CATE has the highest three-timestep average in all four model/subset settings, improves substantially over leaving skills stale, and uses fewer adaptation tokens and logical calls than ContDa in the reported longer-chain comparison.

Critical strengths
  1. The motivation is reasonable and receives direct experimental support. DynaToolBench keeps tasks executable while changing agent-visible API names, arguments, and outputs, so stale-interface failures can be attributed to adaptation. Table 3 shows that on 8B/Dyna-Long, leaving skills unchanged after replacement yields 68.61% and 67.78% at T1/T2, below current Raw-Doc's 77.22% and 73.33%. This concretely demonstrates the problem the paper proposes to address. The benchmark's short and longer dependency chains also test changes beyond isolated tool selection.
  2. The method responds plausibly to localized interface invalidation and demonstrates recovery. CATE identifies affected skill positions, grounds replacements and inherited notes in current documentation, and preserves unaffected entries. Full CATE recovers to 78.47% and 79.17% in the same Table 3 setting. Raw-Doc, EasyTool, and the continual ContDa baseline are relevant comparisons, and lower transition-time update cost than ContDa supports the efficiency motivation. The case study makes local workflow repair concrete, although these results do not isolate experience inheritance itself.

The inspected code substantiates this local-update mechanism: composite repair replaces only retired API positions while retaining general skill fields, atomic inheritance appends selected transferable notes, and frozen evaluation checks artifact hashes. Deterministic fixtures confirm these implementation properties. This strengthens confidence that the described mechanism is implemented, while leaving its incremental performance contribution to be established experimentally.

Critical weaknesses
  • The implemented inheritance mechanism is clear, but its incremental contribution is not isolated. core/atomic_replacement_migration.py actually appends selected old notes to new atomic skills, while core/replacement_adaptation.py uses current documents to replace affected composite positions. This confirms that reuse is implemented. However, document-guided replacement could account for much of the recovery without useful old experience. Table 3 removes all evolution or all adaptation; it does not compare inheritance with the same skill structure and update procedure after old notes are cleared. Raw-Doc/EasyTool lack that structure, and ContDa changes representation and adaptation together. Separate matched controls for atomic inheritance, composite skills, and reflection remain necessary to attribute the gains to selective reuse of accumulated experience.
  • Implemented retention and success indicators are not yet sufficient experimental evidence for the two stated objectives. The code preserves unaffected skill entries and protected fields, which is meaningful implementation evidence; it does not imply unchanged task behavior after retrieval or environment updates. The 94 tasks span two solution-preserving transitions, but DL has 28/30 affected tasks at T1 and all 30 at T2, leaving little direct evidence for retention. I also refine the earlier metric concern: execution/evaluator.py computes both mean requirement accuracy and success only when all requirement scores equal 1, and the runner aggregates complete-task success and invocation diagnostics. The issue is therefore whether the reported comparative conclusions hold on these stricter measures, not whether they are implemented. Aggregate requirement accuracy and a three-timestep average can conceal incomplete downstream goals or adaptation regressions; for example, 8B/DS at T2 remains 72.66% for CATE versus 74.48% for Raw-Doc. Separate affected/unaffected pre/post outcomes and the already available complete-task diagnostics are needed to show that the method solves both recovery and retention.
  • Writing quality: frequent defensive statements and excessive qualification of the method and evaluation. The paper repeatedly emphasizes preserved backend functionality and task solvability, then stresses that composite skills encode local relations instead of complete task solutions, retrieved skills are guidance “rather than rigid programs”, and adaptation receives “neither benchmark ground-truth trajectories nor formal evaluation outcomes” (Sections 4–6). Additional passages repeatedly separate learning tasks, training-time judgments, and formal evaluation. The cumulative presentation is overly defensive: successive explanations of what the method does not assume or use can make the prose sound like preemptive rebuttal and create an impression of protesting too much. This reduces concision and obscures the central explanation of why inherited experience improves adaptation. This is the consequence of the author's excessive reliance on GPT writing, and the author should optimize these writing sections to make their paper statements more persuasive, thereby improving the quality of their writing.
Questions for authors
  • Using the supplied implementation, does inheritance outperform clearing old notes and rewriting from current documents when skill structure, retrieval, executor, and update budget are held fixed? What remains after separately disabling atomic-note transfer, composite skills, or reflection? These controls should isolate the reuse mechanism rather than only compare whole systems.
  • What are the pre/post-adaptation outcomes separately for affected and unaffected tasks? Does preservation of unrelated skill entries in the code translate into preserved task success, and do previously successful affected tasks recover?
  • The evaluator already computes benchmark_success_rate/end_to_end_success_rate and ground-truth invocation diagnostics. Do the reported CATE gains persist on these complete-task and identifier/dependency measures? How much disagreement is there between averaged requirement accuracy and complete completion, and which failure types explain remaining regressions?
8 comments
AuthorsOpenReview ↗

Author Response to Reviewer 7ncc (Part 1/4): Skill Components and Ablations

Thank you for your detailed feedback! We address the questions about CATE's individual components, task-level recovery and retention, and evaluation metrics below. We also outline the planned writing revisions.

W1 & Q1. Skill Composition, Experience Inheritance, and Reflection

You asked how much composite Skills, atomic experience inheritance, and reflection each contribute to CATE. We clarify their roles and report two additional ablations.

Role of Composite Skills. Composition is performed during Skill Evolution at T0. Composite Skills do more than represent inter-tool dependencies: they also provide the interactions from which atomic Skills learn. CATE generates diagnostic tasks based on discovered tool compositions, executes them in the environment, and propagates reusable tool-specific experience back to the participating atomic Skills.

Simply removing composition would therefore also remove the main source of interaction evidence for atomic Skill learning. The remaining atomic Skills would largely rely on documentation, making this setting similar to the Raw-Doc baseline.

Additional Ablations. We also evaluated w/o Atomic Inheritance and w/o Reflection to isolate the contributions of experience transfer and interaction-guided refinement.

Table 1. Ablation results on Qwen3-8B / Dyna-Long (Accuracy, %).

MethodT0T1+T2+Avg.
Full CATE81.6778.4779.1779.77
w/o Adaptation81.6768.6167.7872.69
w/o Atomic Inheritance81.6773.0675.6976.81
w/o Reflection75.5677.2277.2276.67

The identical T0 scores (81.67%) for Full CATE, w/o Adaptation, and w/o Atomic Inheritance reflect a shared initial T0 SkillBank and evaluation, since these ablations affect only subsequent adaptation stages.

The w/o Atomic Inheritance variant retains the SkillBank structure and document-guided composite adaptation but disables the transfer of accumulated atomic experience. Accuracy drops by 5.41 pp at T1 and 3.48 pp at T2, indicating that inherited experience contributes beyond document-guided interface replacement.

Removing Reflection reduces T0 accuracy by 6.11 pp, showing the benefit of refining Skills through interaction feedback.

These results show that reflection improves the acquired experience, while atomic inheritance carries it across API replacements.

AuthorsOpenReview ↗

Author Response to Reviewer 7ncc (Part 2/4): Task Recovery and Retention

W2 & Q2. Affected-Task Recovery and Unaffected-Task Retention

You correctly point out that preserving Skill entries does not necessarily mean preserving task performance. We therefore examined what happens to individual tasks before and after adaptation, using one complete CATE run for each of Qwen3-4B and Qwen3-8B on Dyna-Short (DS) and Dyna-Long (DL).

Pre-/Post-Adaptation Accuracy. We first measured requirement-level accuracy on affected and unaffected tasks within the same updated environment, immediately before and after adaptation.

Table 2. Pre-/post-adaptation Accuracy (%) on affected and unaffected tasks.

ModelSubsetStageAffected (Pre → Post)Unaffected (Pre → Post)
4BDST170.00 → 74.17 (+4.17)79.17 → 79.17 (±0.00)
4BDST272.50 → 80.00 (+7.50)61.11 → 61.11 (±0.00)
4BDLT176.49 → 84.23 (+7.74)100.00 → 75.00 (−25.00)
4BDLT277.38 → 78.27 (+0.89)100.00 → 100.00 (±0.00)
8BDST171.88 → 80.62 (+8.74)64.58 → 64.58 (±0.00)
8BDST271.88 → 75.00 (+3.12)70.14 → 68.75 (−1.39)
8BDLT169.20 → 78.12 (+8.92)83.33 → 83.33 (±0.00)
8BDLT276.79 → 77.68 (+0.89)50.00 → 100.00 (+50.00)

Changes in parentheses are percentage points (pp). Affected/unaffected tasks are identified separately for each environment transition. Each transition contains 40 affected and 24 unaffected tasks on DS, and 28 affected and only 2 unaffected tasks on DL. Thus, both the −25.00 pp and +50.00 pp changes on DL concern groups of just two tasks, rather than reflecting broad performance changes. Although all 30 DL tasks are affected cumulatively across both transitions, two remain unaffected by each individual transition.

Affected-task accuracy increases after adaptation in all eight settings. For unaffected tasks, accuracy remains unchanged in five settings. The other three show changes, including relatively large percentage fluctuations in the two-task DL groups. We report these cases explicitly rather than treating unchanged Skill entries as a guarantee of unchanged task outcomes.

Complete-Task Recovery and Retention. We further checked whether individual tasks returned to full completion after adaptation. Here, full completion requires every evaluated requirement to receive a score of 1.

Table 3. Recovery and retention of fully completed tasks.

ModelSubsetStageAffected RecoveryUnaffected Retention
4BDST12/5 (40.0%)15/15 (100%)
4BDST25/7 (71.4%)8/11 (72.7%)
4BDLT13/3 (100%)1/2 (50.0%)
4BDLT25/6 (83.3%)2/2 (100%)
8BDST14/6 (66.7%)11/14 (78.6%)
8BDST24/5 (80.0%)13/14 (92.9%)
8BDLT14/4 (100%)1/1 (100%)
8BDLT25/6 (83.3%)1/1 (100%)

Affected Recovery measures the fraction of previously fully completed affected tasks that became incomplete following API replacement but returned to full completion after adaptation. Unaffected Retention measures the fraction of unaffected tasks fully completed immediately before adaptation that remained fully completed afterward.

Across both transitions, Qwen3-4B recovers 15/21 (71.4%) and Qwen3-8B recovers 17/21 (81.0%) of previously successful but disrupted task-completion events. For unaffected tasks, 26/30 (86.7%) previously successful task-transition cases remain fully completed for each model. These counts refer to task-transition cases, not necessarily distinct tasks.

The two measurements address different aspects of adaptation: requirement-level accuracy captures partial recovery, while complete-task outcomes show how often previously successful tasks are fully restored or retained. The results also show that retention is not perfect, particularly in some individual transitions.

AuthorsOpenReview ↗

Author Response to Reviewer 7ncc (Part 3/4): Writing Revisions and Complete-Task Success

W3. Writing and Presentation

We agree with your criticism that parts of Sections 4–6 are repetitive and occasionally defensive. Several qualifications were intended to make the experimental setting precise, but they made the main argument harder to follow. We will revise these sections accordingly.

In Section 4, we will remove repeated explanations of backend stability and task solvability, keeping the focus on how DynaToolBench constructs controlled API changes.

In Section 5, we will remove unnecessary qualifications such as “rather than rigid programs” and describe Skill composition, interaction-guided refinement, and experience inheritance more directly.

In Section 6, we will reduce repeated explanations of the distinction between Skill learning and evaluation, while retaining the necessary protocol details and placing longer technical clarifications in the appendix.

The aim is to make the method and experimental setting easier to follow without removing information needed for reproducibility.

Q3. Complete-Task Success and Tool-Call Dependency Diagnostics

We also examined complete-task success, adherence to reference tool dependencies, and the remaining failure cases. The analyses cover Qwen3-4B and Qwen3-8B, including the additional AWM and ReasoningBank baselines.

Complete-Task Success. Our primary metric is requirement-level Accuracy because it distinguishes partial completion from complete failure. In a dependent workflow, an agent may correctly perform several operations but fail on a later step. Binary success assigns such cases the same outcome as tasks with no completed requirements. We therefore report complete-task success as an additional, stricter metric.

Table 4. Three-timestep average complete-task success rate (%).

Method4B/DS4B/DL8B/DS8B/DL
Raw-Doc60.9444.4461.9850.00
EasyTool53.1342.1257.8143.33
ContDa58.8545.5658.8551.78
AWM56.7742.2255.2147.78
ReasoningBank61.4636.6759.9045.56
CATE58.8550.0063.0255.56

CATE ranks first in three of the four settings, exceeding the strongest baseline by 4.44 pp on 4B/DL, 1.04 pp on 8B/DS, and 3.78 pp on 8B/DL. On 4B/DS, ReasoningBank achieves higher complete-task success, although CATE has higher requirement-level Accuracy.

The gaps between requirement-level Accuracy and complete-task success are also substantial across all methods: 20.15 pp for Raw-Doc, 19.60 pp for EasyTool, 21.24 pp for ContDa, 22.71 pp for AWM, 21.95 pp for ReasoningBank, and 20.61 pp for CATE. The gaps are generally larger on Dyna-Long, where completing every requirement in a dependent workflow is more difficult. This also explains why requirement-level Accuracy and binary complete-task success can give different rankings.

Reviewer_z1WMOpenReview ↗
Rating
2
Confidence
3
Informativeness
3(+1)
Presentation
2
Soundness
2
Read review
Paper summary

This paper mainly studies dynamic tool learning, that is, agents must retain unaffected tool-use capabilities while efficiently adapting behaviors that have become invalid due to environmental changes. The authors propose the CATE framework, which organizes tool-use experience into an evolving population of skills. CATE represents accumulated tool-use experience as reusable skills that cover not only the use of individual tools but also the sequential and dependency relationships among tools. The authors also construct DynaToolBench, which fixes the backend and changes tool interfaces to evaluate adaptability under the condition that tasks remain solvable.

Critical strengths
  1. Composite Skills are nominated as candidates by an LLM from the current closed Tool Pool, and a second LLM call performs dependency/quality gating. The tool sequence is programmatically fixed, and new skills must also generate diagnostic tasks that are executed in the real environment and written back to fitness. This makes the method inspectable and reproducible.
  2. The paper organizes tool-use experience into Skills consisting of ordered tool sequences plus procedural experience, rather than scattered tool documentation; when the environment changes, it only identifies and revises the Skills containing the replaced API, preserving fields such as title, principle, and when-to-apply.
Critical weaknesses
  1. The paper does not prove that it is superior to a simple strategy: matching old and new tools, and then directly patching names, parameters, and output fields in the old experience. If simple patching can achieve similar results, the complex skill evolution mechanism lacks necessity.
  2. In Table 2, CATE's average advantages over the strongest baseline in each setting are +1.11, +3.61, +0.61, and +1.38 percentage points, respectively; DS has only 64 tasks, and DL has only 30 tasks. It is impossible to determine whether the small advantages are stable.
  3. Figures 4 and Table 3 provide evidence of overall performance recovery after adaptation, but the existing results do not distinguish between the recovery of affected tasks and performance changes on unaffected tasks. Therefore, an improvement in the overall score alone is insufficient to support the claim that effective capabilities are preserved.
Questions for authors

NA

4 comments
AuthorsOpenReview ↗

Author Response to Reviewer z1WM (Part 1/2): Adaptation and Benchmark Evaluation

Thank you for your constructive feedback and valuable suggestions! Your comments have helped us identify important opportunities to clarify our adaptation mechanism and strengthen the empirical evaluation. We address your concerns below.

W1. Necessity and Simplicity of CATE's Adaptation

We recognize your point that adaptation to API interface changes might be achievable through straightforward patching. We would like to clarify that CATE's adaptation is itself a lightweight, localized patching procedure, rather than a repeated execution of the full Skill Evolution mechanism. The evolution process is performed only once at T0 to construct the initial SkillBank. At T1 and T2, CATE identifies Skills affected by API replacements, updates the corresponding tool bindings and execution guidance, and inherits reusable atomic experience. No additional expansion, reflection, or selection is performed during adaptation.

Thus, the key contribution is not a complex adaptation algorithm, but the combination of reusable dependency-aware Skills learned at T0 and efficient, targeted knowledge updates under environment changes. The existing SkillBank enables CATE to preserve accumulated procedural knowledge while modifying only the interface-dependent components.

Our additional ablations further support this design. Removing atomic experience inheritance decreases accuracy by 5.41 and 3.48 percentage points at T1 and T2, respectively, while removing reflection reduces T0 accuracy by 6.11 points. These results demonstrate the value of accumulated and inherited experience beyond interface replacement alone.

W2. Benchmark Scale, Evaluation Reliability, and Performance Consistency

We take seriously your concerns about the benchmark size and the consistency of the reported improvements. To provide more comprehensive evidence, we examine the benchmark construction process, evaluation reliability, and performance across additional baselines and backbone models.

Benchmark Scale and Quality. DynaToolBench is comparable in scale to individual evaluation subsets of established tool-use benchmarks. For example, RestBench contains 100 TMDB and 57 Spotify tasks, while τ-bench includes 50 Airline and 115 Retail tasks. StableToolBench also includes 61 solvable G3 Instruction tasks. However, these existing benchmarks are not specifically designed to evaluate continual adaptation under evolving API interfaces, which is the central focus of our study.

Our DynaToolBench comprises 94 carefully curated tasks (64 Dyna-Short and 30 Dyna-Long), evaluated across three successive environment versions. Its construction follows a multi-stage quality-control pipeline: among 518 generated tasks, 217 passed the five-dimensional quality assessment, and 94 were ultimately retained through human review. The review considered task naturalness, semantic coherence, meaningful inter-tool dependencies, task diversity, and API coverage. This process prioritizes executable, high-quality tasks with meaningful multi-step tool interactions.

Fine-Grained Evaluation and Reliability. Importantly, our evaluation goes beyond binary task success. Each user query is decomposed into semantically independent requirements, with each requirement scored as 0, 0.5, or 1 based on actual API calls, tool observations, and final responses. This enables fine-grained assessment of task completion, including partial fulfillment, incorrect tool selection, and failures in identifier propagation across dependent API calls.

We additionally validated Gemini's evaluations through three independent annotators on 128 execution trajectories, comprising 100 stratified random samples and 28 challenging cases. Using an absolute task-score difference threshold of 0.25 between Gemini and the annotators' mean scores, the agreement rate reached 82.0% (105/128).

AuthorsOpenReview ↗

Author Response to Reviewer z1WM (Part 2/2): Additional Experiments and Task-Level Analysis

W2: Additional Experimental Evidence (continued)

Additional Experimental Evidence. To further address your concern about performance consistency, we expanded our experiments with two additional experience-based baselines, AWM and ReasoningBank, and introduced Llama-3.1-8B-Instruct to examine another model family.

Table 1. Three-timestep average Accuracy (%) across expanded experimental settings.

BackboneSubsetRaw-DocEasyToolContDaAWMReasoningBankCATE
Qwen3-4BDS72.6568.9773.8972.7973.1875.00
Qwen3-4BDL73.9868.5676.3971.3968.5280.00
Qwen3-8BDS74.4869.0171.3173.4274.0075.09
Qwen3-8BDL76.8568.2478.3975.2375.6979.77
Llama-3.1-8BDS67.1465.0665.2868.1466.8870.23
Llama-3.1-8BDL74.9169.7272.8272.6971.9477.64

Across all six backbone–subset settings, CATE achieves the highest three-timestep average accuracy. Notably, on the newly introduced Llama-3.1-8B-Instruct model, CATE achieves 70.23% on DS and 77.64% on DL, outperforming the strongest competing baselines by 2.09 and 2.73 percentage points, respectively.

These expanded comparisons show that CATE's performance advantages extend beyond the original baselines and Qwen model family. Together with the carefully curated benchmark, fine-grained evaluation protocol, and independent assessment, they provide broader empirical support for the consistency and reliability of our findings.

W3. Affected and Unaffected Task Performance

You raise an important distinction between recovering performance on affected tasks and preserving actual task performance on unaffected tasks. We agree that maintaining unchanged Skill entries alone is insufficient to demonstrate behavioral retention. To address this concern, we conducted an additional analysis using one complete CATE run for each of Qwen3-4B and Qwen3-8B on both Dyna-Short (DS) and Dyna-Long (DL).

We separately evaluated affected and unaffected tasks before and after adaptation within the same updated environment.

Table 2. Pre-/post-adaptation Accuracy (%) on affected and unaffected tasks.

ModelSubsetStageAffected (Pre → Post)Unaffected (Pre → Post)
Qwen3-4BDST170.00 → 74.17 (+4.17)79.17 → 79.17 (±0.00)
Qwen3-4BDST272.50 → 80.00 (+7.50)61.11 → 61.11 (±0.00)
Qwen3-4BDLT176.49 → 84.23 (+7.74)100.00 → 75.00 (−25.00)
Qwen3-4BDLT277.38 → 78.27 (+0.89)100.00 → 100.00 (±0.00)
Qwen3-8BDST171.88 → 80.62 (+8.74)64.58 → 64.58 (±0.00)
Qwen3-8BDST271.88 → 75.00 (+3.12)70.14 → 68.75 (−1.39)
Qwen3-8BDLT169.20 → 78.12 (+8.92)83.33 → 83.33 (±0.00)
Qwen3-8BDLT276.79 → 77.68 (+0.89)50.00 → 100.00 (+50.00)

Changes in parentheses are percentage points (pp). Each transition contains 40 affected and 24 unaffected tasks on DS, and 28 affected and only 2 unaffected tasks on DL. Consequently, both the −25.00 pp change on Qwen3-4B/DL/T1 and the +50.00 pp change on Qwen3-8B/DL/T2 arise from the same very small unaffected-task group size (n=2), rather than reflecting broad performance changes.

Across all eight settings, affected-task accuracy consistently improves after adaptation, demonstrating performance recovery across both backbone models and successive environment transitions.

For unaffected tasks, accuracy remains unchanged in five settings, with smaller changes on DS and larger percentage fluctuations in the two-task DL groups. These observations complement CATE's structural preservation of unrelated Skills with direct measurements of task-level performance.

Together, these results distinguish recovery on affected tasks from behavioral changes on unaffected tasks, providing additional empirical evidence for CATE's localized adaptation beyond aggregate accuracy.

Paper Revisions

We will incorporate the corresponding clarifications and additional experimental evidence into the revised paper and appendix, including additional ablation results, expanded baseline and backbone comparisons, benchmark construction and evaluation reliability analyses, and detailed affected/unaffected task results.

Thank you again for your thoughtful comments and valuable suggestions!

AuthorsOpenReview ↗

Revised Manuscript Uploaded — Adaptation Mechanism and Experimental Evidence

Thank you for your constructive feedback and valuable suggestions!

We have uploaded a revised PDF incorporating the updates discussed in our rebuttal, with substantive changes highlighted in blue. Although many passages have been revised, the main paper remains nine pages long, as we removed repetitive explanations and used the space to incorporate additional experimental evidence and analyses.

The main updates include clarifications of additional ablation results (Section 6.3, Table 3, and Appendix B.10), expanded baseline and backbone comparisons (Section 6.2, Table 2, and Appendix B.9), and separate analyses of affected-task recovery and unaffected-task retention (Section 6.3, Table 4, and Appendix B.12, Tables 13–14).

Thank you again for your guidance on clarifying the method and strengthening the experimental evidence for adaptation and retention!

Thank you for the author's detailed response. I have raised the Informativeness score to 3.

Reviewer_koJ3OpenReview ↗
Rating
2(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper studies dynamic tool learning scenario, where an LLM agent builds up tool-use knowledge, and then some APIs are replaced by functionally equivalent ones with different names, parameters, or output fields. The paper proposes CATE, a training-free framework that that maintains and evolves a SkillBank over time steps. Every API gets an atomic Skill, then expanded into pairwise composite Skills, with LLM-proposed and LLM-verified dependencies. Skills are refined with self-generated diagnostic tasks and periodic reflection. When the environment changes, CATE finds the Skills that contain removed APIs. For each one, it retrieves the top-3 similar new APIs, and picks the replacement and rewrites the tool guidance once. New atomic Skills can also inherit notes from retired, similar atomic Skills. The paper further builds DynaToolBench from StableToolBench G2/G3 APIs on the MirrorAPI backend, consisting of two subsets with different time steps. Three timesteps apply cumulative, solution-preserving API replacements and a hidden versioned adapter maps each agent-facing interface onto the same backend. On Qwen3-4B and Qwen3-8B, CATE has the highest mean accuracy across the three timesteps in all four model/subset settings. It also uses fewer adaptation tokens and calls than ContDa.

Critical strengths
  1. Benchmark is controlled cleanly. The versioned adapter over a fixed MirrorAPI backend separates interface change from backend instability and from tasks becoming unsolvable. The solution-preserving constraint means a drop in performance can be blamed on outdated knowledge. Building queries from verified, executed chains (rather than generating queries first) and filtering on five quality dimensions are good practices.
  2. Localized adaptation is intuitive and cheap. Rewriting only the affected Skills, while protecting the title, principle, and unaffected positions, is a reasonable design. The cost comparison with ContDa supports the efficiency claim.
  3. Overall best performance for two models, and ablation shows the effectiveness of the method, while costing less tokens then ContDa.
  4. Implementation details are provided.
Critical weaknesses
  1. The paper claims broader than what it has actually done. The paper claims dynamic tool learning scenarios, while what it has tested are visible surfaces. The tool names, parameters, and output fields. Those are all related to how agent calls tools. The semantics, returned values, error behaviors, latency, and cost are all identical, which in real scenarios could change.
  2. The paper lacks comparison with previous skills and self-envolving studies. The method distills agent experiences into a bank, which many previous has done, like AWM, ExpeL, ReasoningBank, ToolExpNet, ToolEVO. What is probably new is how skillbank is formulated, like dependency-indexed localization and atomic inheritance from retired tools. But similar ideas are also presented from previous studies. For example, ContDa has a step that retrieves similar previous tools for later time steps, RAPIM recommends method mappings between libraries using similarity of method signatures and documentation.
  3. Reported improvements are small and are not convincing. The benchmark contains 64 and 30 tasks in each subset respectively. While in such small benchmark, performance improvement is not consistent, and many reported gaps are around 1 point. No multiple runs and variance is reported.
  4. Generality is limited. Experiments are only done on its own constructed benchmark and Qwen3-4B and 8B, with reasoning disabled. What about other benchmarks, families of models with different sizes? Also, it only compare 3 baselines, which is Weakness 2.

[1] ExpeL: LLM Agents Are Experiential Learners

[2] ReasoningBank: Scaling Agent Self-Evolving with Reasoning Memory

[3] ToolExpNet: Optimizing Multi-Tool Selection in LLMs with Similarity and Dependency-Aware Experience Networks

[4] Learning Evolving Tools for Large Language Models

Questions for authors
  1. What fraction of the benchmark’s ground-truth adjacent API pairs are present in the T0 SkillBank? How large is the SkillBank (atomic vs. composite, active vs. deactivated) at each timestep?
  2. The benchmark used Gemini-3.8-Flash for evaluation. How reliable it is? Is it possible to show that the evaluation is confident and reliable (e.g. Human Annotation)? What exactly did the “Human Review” step in Fig. 2(a) involve?
4 comments
AuthorsOpenReview ↗

Author Response to Reviewer koJ3 (Part 1/3): Motivation and Additional Baselines

Thank you for your constructive feedback and thoughtful questions! Your comments have helped us identify where the motivation, methodological distinctions, and empirical evidence need to be communicated more clearly. We address your concerns and questions below.

W1 & W2: Motivation and Novelty

We understand your concerns regarding the practical significance of interface-level tool evolution and the distinction between CATE and existing skill-learning approaches. We agree that our original presentation did not sufficiently emphasize the implications of interface changes for multi-step tool use or clearly articulate our contributions. We will revise the corresponding discussions in the paper to make the research scope, motivation, and methodological distinctions more explicit.

Regarding motivation, we acknowledge that tool evolution extends beyond interface changes. Nevertheless, even when the underlying functionality remains unchanged, modifications to API names, parameters, or output fields can invalidate previously successful multi-step workflows by breaking dependencies between tools. This creates a meaningful continual adaptation challenge: agents must update outdated tool-use knowledge while preserving experience that remains valid. By maintaining task solvability and a stable MirrorAPI backend, our controlled setting isolates this challenge from confounding factors such as capability loss and execution instability.

Regarding novelty, while prior studies have explored skill learning, tool dependencies, and API matching, CATE goes beyond maintaining a collection of independently learned Skills. It takes individual tools as atomic building blocks and emphasizes learning their compositional dependencies and accumulating reusable interaction experience across them. Crucially, this experience is not confined to isolated Skills: reusable knowledge is propagated between composite and atomic Skills and inherited when APIs are replaced. Our central contribution is therefore the continual evolution and adaptation of interdependent tool-use experience, rather than merely storing, refining, or retrieving individual Skills.

W2–W4: Additional Experiments

To further strengthen our empirical evaluation, we conducted additional experiments with two experience-based baselines and an additional backbone model.

1. Additional Baselines (W2). Following your suggestion, we included AWM and ReasoningBank. AWM extracts reusable workflows from agent experience, while ReasoningBank distills generalizable reasoning strategies from successful and failed trajectories. Both provide meaningful comparisons for evaluating CATE's structured skill evolution and adaptation.

Method4B-DS4B-DL8B-DS8B-DL
AWM72.7971.3973.4275.23
ReasoningBank73.1868.5274.0075.69
ContDa73.8976.3971.3178.39
CATE75.0080.0075.0979.77

Accuracy (%), averaged across three timesteps. Following the same protocol as our main experiments, results are averaged over three independent runs.

Both new baselines perform competitively on Dyna-Short, demonstrating the effectiveness of reusable workflows and reasoning memories for relatively short tool-use tasks. However, their performance is less consistent under API replacements, with CATE showing more pronounced advantages on Dyna-Long. General-purpose experience memories capture reusable patterns but do not explicitly model inter-tool dependencies or identify which parts of the experience require updating when interfaces change. In contrast, CATE organizes atomic and composite Skills around tool dependencies and selectively adapts affected Skills, helping preserve their utility across evolving environments.

AuthorsOpenReview ↗

Author Response to Reviewer koJ3 (Part 2/3): Additional Backbone and Benchmark

W2–W4: Additional Experiments (continued)

2. Additional Backbone (W4). You also raised concerns about generalization beyond the Qwen family. We therefore evaluated Llama-3.1-8B-Instruct against all five baselines.

MethodDyna-ShortDyna-Long
Raw-Doc67.1474.91
EasyTool65.0669.72
ContDa65.2872.82
AWM68.1472.69
ReasoningBank66.8871.94
CATE70.2377.64

Accuracy (%), averaged across three timesteps.

CATE achieves the best results on both subsets, outperforming the strongest baselines by 2.09 and 2.73 percentage points, respectively. Notably, AWM is the strongest baseline on Dyna-Short, while Raw-Doc leads on Dyna-Long, suggesting that different experience-utilization strategies have varying effectiveness across task complexities. CATE's consistent advantage across both subsets further supports the transferability of its skill evolution and localized adaptation mechanisms beyond the Qwen family.

Together, these experiments provide broader evidence for CATE's effectiveness across different baselines, task complexities, and model families. We further address your concerns regarding benchmark scale, construction quality, and evaluation reliability below.

W3 & W4: Benchmark Construction and Evaluation

We understand your concerns regarding benchmark scale and generalizability. We clarify two aspects of our benchmark design.

1. Benchmark Suitability. Existing tool-use benchmarks, such as RestBench and τ-bench, provide valuable evaluations of agent capabilities, but are not specifically designed to assess continual adaptation to evolving API interfaces. Evaluating CATE directly on these benchmarks would therefore not capture our central research question: whether agents can preserve and update accumulated tool-use knowledge as their environments change. This motivates DynaToolBench, which introduces controlled API replacements across three timesteps while preserving task semantics and backend functionality, enabling a focused evaluation of knowledge evolution and adaptation.

2. Construction Quality, Scale, and Evaluation. DynaToolBench follows a multi-stage construction process. Starting from StableToolBench G2/G3, we validate executable APIs, identify meaningful inter-API dependencies through semantic and execution verification, and generate queries only from successfully executed tool chains. Candidates are then assessed along five dimensions—naturalness, coherence, necessity, dependency quality, and solvability—followed by a final manual review. This process prioritizes coherent, executable, dependency-grounded tasks over artificially expanding the dataset.

The resulting benchmark contains 64 Dyna-Short and 30 Dyna-Long tasks, comparable in scale to individual evaluation subsets of established tool-use benchmarks, such as RestBench–Spotify (57 tasks) and τ-bench–Airline (50 tasks). More importantly, DynaToolBench prioritizes task quality, executable dependencies, and controlled environment evolution over simply increasing the number of tasks.

Moreover, unlike conventional binary success evaluation, our requirement-level evaluation protocol assigns scores of 0, 0.5, or 1 to individual user-facing requirements, based on actual tool executions, observations, and final responses. This distinguishes complete, partial, and failed fulfillment, providing more informative and auditable measurements of agent performance.

Together, the controlled environment construction, task filtering, and fine-grained evaluation provide a focused testbed for studying adaptation in evolving tool environments.

AuthorsOpenReview ↗

Author Response to Reviewer koJ3 (Part 3/3): SkillBank Statistics and Evaluation Reliability

Q1: SkillBank Statistics and Ground-Truth Pair Coverage

We provide the SkillBank statistics and ground-truth adjacent API-pair coverage for both Qwen3-4B and Qwen3-8B on Dyna-Short (DS) and Dyna-Long (DL).

Table 1. T0 SkillBank statistics and ground-truth API-pair coverage.

ModelSubsetAtomicCompositeTotalGT-Pair Coverage
Qwen3-4BDS509914916/64 (25.00%)
Qwen3-4BDL347410813/39 (33.33%)
Qwen3-8BDS506711710/64 (15.63%)
Qwen3-8BDL346810210/39 (25.64%)

Coverage is calculated using distinct ground-truth adjacent API pairs and exact matches with the ordered API sequences in T0 composite Skills.

Across T0, T1, and T2, the Atomic and Composite Skill counts remain unchanged in all four settings. After the initial Skill Evolution at T0, CATE handles subsequent environment changes through localized updates to affected tool bindings and execution guidance, together with inheritance of atomic experience. This maintains the existing SkillBank without repeating the evolution process.

Importantly, CATE does not aim to enumerate the API pairs appearing in benchmark ground-truth trajectories or maximize their exact coverage. With a tool pool of N APIs, the number of possible ordered API pairs grows quadratically, making exhaustive coverage impractical under a limited evolution budget, particularly in realistic environments with large and diverse API ecosystems. More fundamentally, such enumeration is not the objective of experience learning.

Instead, CATE aims to acquire reusable knowledge about individual tool usage, meaningful inter-tool dependencies, and procedural execution experience, including argument construction, output interpretation, and information flow between tools. These learned Skills can provide useful guidance for new multi-step workflows even when their exact API pairs have not been encountered during evolution. Thus, the value of the SkillBank lies not merely in exact pair coverage, but in the transferability and reuse of accumulated tool-use experience.

Q2: Human Review and Evaluation Reliability

Human Review. DynaToolBench follows a multi-stage quality-control pipeline combining execution verification, LLM-based quality assessment, and human review. Starting from 518 generated tasks, 217 passed the five-dimensional quality assessment, and 94 were ultimately retained through human review (64 Dyna-Short and 30 Dyna-Long).

The human review focused on four criteria: (1) task naturalness and semantic coherence, (2) meaningful and necessary inter-tool dependencies, (3) diversity of task types, and (4) coverage of different API compositions and dependency-chain lengths. For example, to prevent repetitive task patterns from dominating Dyna-Long, we limited email-related tasks to 12 and constrained the occurrence of individual APIs and adjacent API pairs. This process produced a compact benchmark with verified executability, meaningful tool dependencies, and diverse multi-step workflows.

Evaluation Reliability. We further validated the Gemini-based evaluation through three independent annotators on 128 agent execution trajectories. These included 100 stratified random samples (50 DS and 50 DL) and 28 challenging cases (15 DS and 13 DL). The sampled trajectories covered Qwen3-4B, Qwen3-8B, and Llama-3.1-8B-Instruct across different methods and evaluation stages. Each trajectory was assessed against the original user requirements, tool calls, execution observations, and final responses.

We compared Gemini's original task-level scores with the mean scores of the three independent annotators, using an absolute difference threshold of 0.25.

Sample TypeDSDLTotalAgreement (≤0.25)
Stratified Random505010080.0% (80/100)
Challenging Cases15132889.3% (25/28)
Overall656312882.0% (105/128)

Overall, 82.0% of Gemini's task-level scores fall within 0.25 of the independent annotators' mean scores. This provides empirical support for the reliability of our requirement-level evaluation protocol.

Paper Revisions

We will incorporate the corresponding clarifications and experimental evidence into the revised paper and appendix, including the refined motivation and novelty discussion, expanded baseline and backbone comparisons, benchmark construction details, SkillBank statistics, and evaluation reliability analysis.

Thank you again for your valuable feedback and suggestions!

AuthorsOpenReview ↗

Revised Manuscript Uploaded — Motivation, Novelty, and Experimental Evaluation

Thank you for your thoughtful and constructive feedback!

We have uploaded a revised PDF incorporating the updates discussed in our rebuttal, with substantive changes highlighted in blue. The main paper remains nine pages long despite the extensive revisions: we removed repetitive and unnecessary explanations to make room for additional experimental evidence and analyses suggested during the review.

The main updates include clarifications of the research scope and novelty (Sections 1–2 and 5), expanded baseline and backbone comparisons (Section 6.2, Table 2, and Appendix B.9), further details on benchmark construction and human review (Section 4 and Appendix A), SkillBank statistics and ground-truth pair coverage (Appendix B.11, Table 12), and evaluation reliability analysis (Section 6.1 and Appendices B.6–B.7, Table 10).

Thank you again for your valuable suggestions on strengthening the paper's motivation, methodological contributions, and empirical evaluation!

Comments on the paper

1 comment
AuthorsOpenReview ↗

Final Summary of Rebuttal and Revisions

Dear Area Chair,

We sincerely thank you and all reviewers for your time, constructive feedback, and thoughtful discussions throughout the review process!

We have carefully addressed the major concerns raised during the discussion and uploaded a revised manuscript. We would like to summarize the key contributions and improvements for your consideration.

1. Core Contributions. CATE addresses continual adaptation to evolving tool interfaces by learning and maintaining reusable, interdependent tool-use experience. Its main contributions include a dependency-aware SkillBank, interaction-guided skill evolution, efficient localized adaptation with experience inheritance, and DynaToolBench, a controlled benchmark for evaluating adaptation across environment changes. Reviewers recognized the value of our benchmark design, localized adaptation mechanism, and computational efficiency.

2. Strengthened Experimental Evidence. In response to the reviewers' suggestions, we conducted extensive additional experiments and analyses:

  • Expanded comparisons: Added AWM, ReasoningBank, and Llama-3.1-8B-Instruct. CATE achieves the highest average accuracy across all six backbone–subset settings.
  • Component ablations: Added atomic inheritance and reflection ablations, demonstrating the contributions of accumulated experience and interaction-guided refinement.
  • Recovery and retention: Analyzed affected and unaffected tasks separately, showing improved affected-task accuracy after adaptation in all eight examined settings.
  • Evaluation reliability and diagnostics: Added strict complete-task success comparisons, tool-dependency analysis, regression diagnostics, and independent human evaluation of 128 trajectories, with 82.0% agreement within a 0.25 score difference.

3. Manuscript Revisions and Reviewer Feedback. We have incorporated these results into the revised manuscript and appendix, clarified our methodological contributions, and substantially improved the writing and figure layouts while maintaining the nine-page main paper. We are particularly grateful to Reviewer 7ncc for carefully reviewing the revised manuscript and expressing support for our work, raising their overall rating from 3 to 4.

We believe these additions substantially strengthen the empirical evidence, clarity, and overall presentation of CATE. We sincerely appreciate your consideration of our revised submission and thank all reviewers again for their valuable feedback and engagement!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.