acceptodds
Under review as a conference paper at ICLR 2027

How Do Tool Surfaces Shape Inference Trajectories in LLM Agents?

Abstract

Tool surfaces, the tools exposed to an agent at each model call, define its available actions and can shape its observable inference trajectory before any tool executes. We investigate their immediate effects on generated reasoning, propagation across calls, and value for execution efficiency. We introduce Tool-Surface-Controlled Trajectory Dynamics (TSCTD), a state-conditioned theoretical framework that distinguishes immediate shaping, historical effects after withdrawal, and effects of continued exposure. Controlled experiments reveal immediate changes in generated reasoning and setting-dependent behavioral carryover after tool withdrawal. They further show that explicit task-closing instructions reduce the additional token cost of continued tool exposure. Building on these findings, we develop a state-dependent configuration policy that uses a compact tool set during execution and withdraws tools once public tests pass. We evaluate this policy on programming problems from the Mostly Basic Python Problems (MBPP) benchmark, adapted into interactive coding and testing tasks. Compared with keeping all tools available throughout execution, the policy reduces aggregate token usage across tasks by 34.74%, while meeting a prespecified success noninferiority criterion. This work establishes tool exposure as a design variable for agent inference and execution and provides an experimental framework for studying when to show, retain, and withdraw tools.

Primary area:
foundation or frontier models, including LLMs
Keywords:
large language models, LLM agents, tool use, tool surfaces, inference trajectories

Reviews

Among:
Shape:
1234this paper, Bottom 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.00 over 3 reviews, above 0% of the others.
Reviewer_EY5NOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper investigates how tool surfaces shape observable agent behavior, and proposes the TSCTD framework to structure the following three research questions:

  • RQ1 (Immediate effects): In some experiments, modifying the list of available tools altered the text generated or the tools selected by the model in the first step. However, this phenomenon was observed only in certain models, and two more rigorously controlled directional experiments failed to yield consistent results; thus, no stable pattern could be established.
  • RQ2 (Post-Withdrawal effects): On some legacy tasks, DeepSeek Pro and Flash continued to call tools that had already been withdrawn, which was not replicated in Qwen and GLM. Furthermore, large experiments—designed to test for propagation, decay, and response-core effects—largely failed, precluding the establishment of any general "carryover" or dynamic behavioral patterns.
  • RQ3 (Cost benefits): In experiments using 192 tasks adapted from the MBPP dataset, withdrawing tools after they had successfully passed public test cases resulted in a 34.74% reduction in token usage compared to a baseline where all seven tools were provided throughout the entire process. However, this approach failed to meet the non-inferiority threshold for success rates when compared to a more robust "static-core" baseline. Additionally, a post-hoc "direct-stop" strategy proved even more cost-effective. Consequently, we could not demonstrate that this tool-withdrawal strategy offers consistent, incremental value.
Critical strengths
  1. The research question posed in the paper is simple and clear: whether different tool surfaces alter the behavior and execution costs of LLM agents. The research objective is straightforward and easy to grasp.
  2. The authors conducted extensive experiments around the research questions and reported positive results, negative results, and results that did not meet the presupposition tests in a relatively complete manner, demonstrating sufficient experimental investment and good reporting transparency.
Critical weaknesses
  1. TSCTD remains an organizational framework rather than a validated theory, and the empirical evidence does not form a coherent conclusion. Although it distinguishes immediate effects, post-withdrawal effects, and sustained-exposure effects, it provides no clear predictions about their direction, magnitude, or patterns. Most pre-planned experiments failed to support the original hypotheses, while results varied across models, tasks, tool-description orderings, and full-execution settings. Together with the extensive post-hoc analyses and failure-driven follow-up experiments, the three research sections appear more like independent explorations around a shared topic than components of a unified methodology, limiting the framework's general utility.
  2. The experimental setting has limited connection to real-world agent systems and lacks broad external validation. The paper does not establish that abrupt tool-surface transitions such as S→M are common in deployment, nor does it measure how ineffective tool calls affect final task outcomes. Its evaluation relies mainly on narrow, constructed tasks rather than realistic agent benchmarks. In my informal TerminalBench test with Claude Opus 5.0, changing tool-description order produced only negligible fluctuations, contrary to RQ1, further motivating controlled evaluation across broader models and benchmarks.

Minor weakness: The main text consists almost entirely of prose, with very few figures or tables. Figure 1, which serves as the primary framework illustration, is composed almost exclusively of text, failing to provide an intuitive visualization of the experimental workflow or the relationships between variables.

Questions for authors

See weaknesses.

Reviewer_XWnHOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This work investigates how tool exposure influences the inference trajectories and action selection of LLM agents, even before receiving execution feedback. The authors argue that tool definitions can affect agents’ action preferences, introduce cross-turn effects after tools are withdrawn, and increase unnecessary reasoning and tool usage after task completion. To systematically investigate these effects, the authors introduce Tool-Surface-Controlled Trajectory Dynamics (TSCTD), a framework that distinguishes immediate effects, historical effects, and continued exposure effects. The authors conduct controlled experiments across coding tasks using models such as DeepSeek-V4, Qwen3.6, and GLM-5.3. Based on the findings, they further propose a state-dependent tool configuration policy that dynamically withdraws tools once public tests pass. Experimental results on MBPP demonstrate a 34.74% reduction in token consumption compared with continuously exposing all tools, while maintaining comparable task success rates.

Critical strengths
  1. Relevant research motivation. The motivation is partially sound. Tool definitions and descriptions can influence an agent’s decision-making, including tool selection, invocation, and task termination. Understanding how tool documentation affects agent behavior is important for designing more reliable and efficient agent systems.
Critical weaknesses
  1. Limited significance and generalizability of the findings. The reported conclusions are relatively weak, and their generalizability to stronger models and real-world agent applications remains unclear. For example, the first research question investigates how tool descriptions and their ordering affect agent behavior. However, it remains unclear (a) why the observed behavioral differences are practically significant and (b) whether these effects primarily arise from the ordering of tool descriptions or their semantic content. Furthermore, the evaluated models are insufficient to establish whether these findings generalize to stronger models, such as recent GPT or Claude models, which may be more robust to variations in tool presentation. Additionally, modern agent harnesses, such as OpenClaw, typically maintain persistent tool configurations rather than repeatedly introducing tools at every execution step. This raises concerns about the practical relevance and applicability of the identified issues.
  2. Insufficient consideration of KV-cache efficiency in the proposed method. The state-dependent tool configuration policy introduced in RQ3 dynamically modifies the available tools, which may disrupt prefix caching and reduce KV-cache reuse across successive model calls. Although the authors report approximately 30–40% reductions in token consumption, these savings do not necessarily translate into equivalent reductions in actual inference costs. In practice, cached input tokens can be substantially cheaper than uncached tokens, sometimes by approximately an order of magnitude. Therefore, dynamically changing the tool configuration may introduce additional costs that offset the reported token savings. Without evaluating cache-aware inference costs, the practical efficiency benefits of the proposed method remain unclear.
Questions for authors
  1. How can the reported findings and the proposed tool configuration strategy be applied to modern agent harnesses, where tool definitions are typically maintained across multiple execution steps? What practical benefits would the proposed approach provide in these settings?
  2. How does the proposed dynamic tool configuration strategy affect KV-cache reuse across successive model calls? Given the substantial pricing difference between cached and uncached input tokens, would the proposed method still achieve meaningful cost reductions when accounting for cache invalidation and actual API pricing?
Reviewer_PrupOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies whether the set and presentation of tools exposed to an LLM agent—the “tool surface”—affect its observable responses, action choices, cross-call behavior, and token consumption. It introduces Tool-Surface-Controlled Trajectory Dynamics (TSCTD), which separates immediate effects of current exposure, historical effects after tool withdrawal, and effects of sustained exposure. The paper reports a large collection of controlled experiments using several models and programming tasks. Some experiments show that expanding the tool inventory changes Qwen’s choice between editing interfaces, while related effects are not consistently observed for other models. The paper also evaluates a state-dependent policy that exposes a compact tool set during execution and removes tools after public tests pass. On an adapted MBPP benchmark, this policy reduces pooled token usage by 34.74% relative to continuous full-tool exposure while satisfying a prespecified success noninferiority criterion. An additional matched-state experiment finds that explicit closing instructions substantially reduce the extra continuation cost associated with retaining tools.

Critical strengths

Careful experimental controls and transparent reporting. The paper distinguishes current exposure, retained interaction state, and continued exposure through matched-state interventions. It also reports failed prespecified tests rather than presenting only positive findings. This transparency improves the credibility of the individual empirical results.

Critical weaknesses

Limited novelty and unclear importance of the central question. The main premise is that changing the tools and tool descriptions included in the model context can change the model’s response. Since tool metadata is part of the prompt, this observation is not surprising.

Questions for authors

None

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.